Testing for causality in reconstructed state spaces by an optimized mixed prediction method

Causes of extreme events revealed by Rényi information transfer

Information-theoretic generalization of Granger causality principle, based on evaluation of conditional mutual information, also known as transfer entropy (CMI/TE), is redefined in the framework of Rényi entropy (RCMI/RTE). Using numerically generated data with a defined causal structure and examples of real data from the climate system, it is demonstrated that RCMI/RTE is able to identify the cause variable responsible for the occurrence of extreme values in an effect variable. In the presented example, the Siberian High was identified as the cause responsible for the increased probability of cold extremes in the winter and spring surface air temperature in Europe, while the North Atlantic Oscillation and blocking events can induce shifts of the whole temperature probability distribution.

Robust inference of causality in high-dimensional dynamical processes from the Information Imbalance of distance ranks

We introduce an approach which allows detecting causal relationships between variables for which the time evolution is available. Causality is assessed by a variational scheme based on the Information Imbalance of distance ranks, a statistical test capable of inferring the relative information content of different distance measures. We test whether the predictability of a putative driven system Y can be improved by incorporating information from a potential driver system X, without explicitly modeling the underlying dynamics and without the need to compute probability densities of the dynamic variables. This framework makes causality detection possible even between high-dimensional systems where only few of the variables are known or measured. Benchmark tests on coupled chaotic dynamical systems demonstrate that our approach outperforms other model-free causality detection methods, successfully handling both unidirectional and bidirectional couplings. We also show that the method can be used to robustly detect causality in human electroencephalography data.

Reliable detection of causal asymmetries in dynamical systems

Knowledge about existence, strength, and dominant direction of causal influences is of paramount importance for understanding complex systems. Current methods deduce ambiguous causal links among different observables from (complex) dynamical systems, if a limited amount of realistic data is used. It is particularly difficult to infer the dominant direction of causal influence for synchronizing systems. Missing is a statistically well defined approach that avoids false positive detection while being sensitive for weak interactions. The proposed method exploits the local inflation of manifolds to estimate upper bounds on the information loss among state reconstructions and tests for the absence of causal influences. Simulated data demonstrates that it is robust to intrinsic noise, copes with synchronization, and tolerates moderate amounts of measurement noise.

Compression complexity with ordinal patterns for robust causal inference in irregularly sampled time series

Distinguishing cause from effect is a scientific challenge resisting solutions from mathematics, statistics, information theory and computer science. Compression-Complexity Causality (CCC) is a recently proposed interventional measure of causality, inspired by Wiener–Granger’s idea. It estimates causality based on change in dynamical compression-complexity (or compressibility) of the effect variable, given the cause variable. CCC works with minimal assumptions on given data and is robust to irregular-sampling, missing-data and finite-length effects. However, it only works for one-dimensional time series. We propose an ordinal pattern symbolization scheme to encode multidimensional patterns into one-dimensional symbolic sequences, and thus introduce the Permutation CCC (PCCC). We demonstrate that PCCC retains all advantages of the original CCC and can be applied to data from multidimensional systems with potentially unobserved variables which can be reconstructed using the embedding theorem. PCCC is tested on numerical simulations and applied to paleoclimate data characterized by irregular and uncertain sampling and limited numbers of samples.

Granger Causality on forward and Reversed Time Series

In this study, the information flow time arrow is investigated for stochastic data defined by vector autoregressive models. The time series are analyzed forward and backward by different Granger causality detection methods. Besides the normal distribution, which is usually required for the validity of Granger causality analysis, several other distributions of predictive errors are considered. A clear effect of a change in the order of cause and effect on the time-reversed series of unidirectionally connected variables was detected with standard Granger causality test (GC), when the product of the connection strength and the ratio of the predictive errors of the driver and the recipient was below a certain level, otherwise bidirectional causal connection was detected. On the other hand, opposite causal link was detected unconditionally by the methods based on the time reversal testing, but they were not able to detect correct bidirectional connection. The usefulness of the backward analysis is manifested in cases where falsely detected unidirectional connections can be rejected by applying the result obtained after the time reversal, and in cases of uncorrelated causally independent variables, where the absence of a causal link detected by GC on the original series should be confirmed on the time-reversed series.

Time-Reversibility, Causality and Compression-Complexity

Detection of the temporal reversibility of a given process is an interesting time series analysis scheme that enables the useful characterisation of processes and offers an insight into the underlying processes generating the time series. Reversibility detection measures have been widely employed in the study of ecological, epidemiological and physiological time series. Further, the time reversal of given data provides a promising tool for analysis of causality measures as well as studying the causal properties of processes. In this work, the recently proposed Compression-Complexity Causality (CCC) measure (by the authors) is shown to be free of the assumption that the "cause precedes the effect", making it a promising tool for causal analysis of reversible processes. CCC is a data-driven interventional measure of causality (second rung on the Ladder of Causation) that is based on Effort-to-Compress (ETC), a well-established robust method to characterize the complexity of time series for analysis and classification. For the detection of the temporal reversibility of processes, we propose a novel measure called the Compressive Potential based Asymmetry Measure. This asymmetry measure compares the probability of the occurrence of patterns at different scales between the forward-time and time-reversed process using ETC. We test the performance of the measure on a number of simulated processes and demonstrate its effectiveness in determining the asymmetry of real-world time series of sunspot numbers, digits of the transcedental number π and heart interbeat interval variability.

Implementation of two causal methods based on predictions in reconstructed state spaces

If deterministic dynamics is dominant in the data, then methods based on predictions in reconstructed state spaces can serve to detect causal relationships between and within the systems. Here we introduce two algorithms for such causal analysis. They are designed to detect causality from two time series but are potentially also applicable in a multivariate context. The first method is based on cross-predictions, and the second one on the so-called mixed predictions. In terms of performance, the cross-prediction method is considerably faster and less prone to false negatives. The predictability improvement method is slower, but in addition to causal detection, in a multivariate scenario, it also reveals which specific observables can help the most if we want to improve prediction. The study also highlights cases where our methods and state-space approaches generally seem to lose reliability. We propose a new perspective on these situations, namely that the variables under investigation have weak observability due to the complex nonlinear information flow in the system. Thus, in such cases, the failure of causality detection cannot be attributed to the methods themselves but to the use of data that do not allow reliable reconstruction of the underlying dynamics.

Causality, dynamical systems and the arrow of time

Using several methods for detection of causality in time series, we show in a numerical study that coupled chaotic dynamical systems violate the first principle of Granger causality that the cause precedes the effect. While such a violation can be observed in formal applications of time series analysis methods, it cannot occur in nature, due to the relation between entropy production and temporal irreversibility. The obtained knowledge, however, can help to understand the type of causal relations observed in experimental data, namely, it can help to distinguish linear transfer of time-delayed signals from nonlinear interactions. We illustrate these findings in causality detected in experimental time series from the climate system and mammalian cardio-respiratory interactions.

Causation and information flow with respect to relative entropy

Recently, a rigorous formalism has been established for information flow and causality within dynamical systems with respect to Shannon entropy. In this study, we re-establish the formalism with respect to relative entropy, or Kullback-Leiber divergence, a well-accepted measure of predictability because of its appealing properties such as invariance upon nonlinear transformation and consistency with the second law of thermodynamics. Different from previous studies (which yield consistent results only for 2D systems), the resulting information flow, say T, is precisely the same as that with respect to Shannon entropy for systems of arbitrary dimensionality, except for a minus sign (reflecting the opposite notion of predictability vs. uncertainty). As before, T possesses a property called principle of nil causality, a fact that classical formalisms fail to verify in many situation. Besides, it proves to be invariant upon nonlinear transformation, indicating that the so-obtained information flow should be an intrinsic physical property. This formalism has been validated with the stochastic gradient system, a nonlinear system that admits an analytical equilibrium solution of the Boltzmann type.

Comparison of six methods for the detection of causality in a bivariate time series

In this comparative study, six causality detection methods were compared, namely, the Granger vector autoregressive test, the extended Granger test, the kernel version of the Granger test, the conditional mutual information (transfer entropy), the evaluation of cross mappings between state spaces, and an assessment of predictability improvement due to the use of mixed predictions. Seven test data sets were analyzed: linear coupling of autoregressive models, a unidirectional connection of two Hénon systems, a unidirectional connection of chaotic systems of Rössler and Lorenz type and of two different Rössler systems, an example of bidirectionally connected two-species systems, a fishery model as an example of two correlated observables without a causal relationship, and an example of mediated causality. We tested not only 20000 points long clean time series but also noisy and short variants of the data. The standard and the extended Granger tests worked only for the autoregressive models. The remaining methods were more successful with the more complex test examples, although they differed considerably in their capability to reveal the presence and the direction of coupling and to distinguish causality from mere correlation.