Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble

The causality measure of partial mutual information from mixed embedding (PMIME) revisited

The measure of partial mutual information from mixed embedding (PMIME) is an information theory-based measure to accurately identify the direct and directional coupling, termed Granger causality or simply causality, between the observed variables or subsystems of a high-dimensional dynamical and complex system, without any a priori assumptions about the nature of the coupling relationship. In its core, it is a forward selection procedure that aims to iteratively identify the lag-dependence structure of a given observed variable (response) to all the other observed variables (candidate drivers). This model-free approach is capable of detecting nonlinear interactions, abundantly present in real-world complex systems, and it was shown to perform well on multivariate time series of moderately high dimension. However, the PMIME presents some inefficiencies in its performance mainly when applied on strongly stochastic (linear or nonlinear) systems as it may falsely detect non-existent relationships. Moreover, and by construction, the measure cannot extract purely synergetic relationships present in a system. In the current work, the issue of false detections is addressed by introducing an improved resampling significance test and a procedure of rechecking the identified drivers (backward revision). Regarding the inability to detect synergetic relationships, the PMIME is further enhanced by checking pairs as candidate drivers for the response variable after having considered all drivers individually. The effects of these modifications are investigated in a systematic simulation study on properly designed systems involving strong stochasticity, regressor terms with synergetic effects, and a system dimension ranging from 3 to 30. The overall results of the simulations indicate that these modifications indeed improve the performance of PMIME and alleviate to a significant degree the issues of the original algorithm. Guidelines for balancing between accuracy and computational efficiency are also given, particularly relevant for real-world applications. Finally, the measure performance is investigated in the study of futures of various government bonds and stock market indices in the period around COVID-19 pandemic.

Causalized convergent cross-mapping and its approximate equivalence with directed information in causality analysis

Convergent cross-mapping (CCM) has attracted increased attention recently due to its capability to detect causality in nonseparable systems under deterministic settings, which may not be covered by the traditional Granger causality. From an information-theoretic perspective, causality is often characterized as the directed information (DI) flowing from one side to the other. As information is essentially nondeterministic, a natural question is: does CCM measure DI flow? Here, we first causalize CCM so that it aligns with the presumption in causality analysis-the future values of one process cannot influence the past of the other, and then establish and validate the approximate equivalence of causalized CCM (cCCM) and DI under Gaussian variables through both theoretical derivations and fMRI-based brain network causality analysis. Our simulation result indicates that, in general, cCCM tends to be more robust than DI in causality detection. The underlying argument is that DI relies heavily on probability estimation, which is sensitive to data size as well as digitization procedures; cCCM, on the other hand, gets around this problem through geometric cross-mapping between the manifolds involved. Overall, our analysis demonstrates that cross-mapping provides an alternative way to evaluate DI and is potentially an effective technique for identifying both linear and nonlinear causal coupling in brain neural networks and other settings, either random or deterministic, or both.

Quantifying the diversity of multiple time series with an ordinal symbolic approach

The main motivation of this paper is to introduce the ordinal diversity, a symbolic tool able to quantify the degree of diversity of multiple time series. Analytical, numerical, and experimental analyses illustrate the utility of this measure to quantify how diverse, from an ordinal perspective, a set of many time series is. We have shown that ordinal diversity is able to characterize dynamical richness and dynamical transitions in stochastic processes and deterministic systems, including chaotic regimes. This ordinal tool also serves to identify optimal operating conditions in the machine learning approach of reservoir computing. These results allow us to envision potential applications for the handling and characterization of large amounts of data, paving the way for addressing some of the most pressing issues facing the current big data paradigm.

Causal inference from cross-sectional earth system data with geographical convergent cross mapping

Causal inference in complex systems has been largely promoted by the proposal of some advanced temporal causation models. However, temporal models have serious limitations when time series data are not available or present insignificant variations, which causes a common challenge for earth system science. Meanwhile, there are few spatial causation models for fully exploring the rich spatial cross-sectional data in Earth systems. The generalized embedding theorem proves that observations can be combined together to construct the state space of the dynamic system, and if two variables are from the same dynamic system, they are causally linked. Inspired by this, here we show a Geographical Convergent Cross Mapping (GCCM) model for spatial causal inference with spatial cross-sectional data-based cross-mapping prediction in reconstructed state space. Three typical cases, where clearly existing causations cannot be measured through temporal models, demonstrate that GCCM could detect weak-moderate causations when the correlation is not significant. When the coupling between two variables is significant and strong, GCCM is advantageous in identifying the primary causation direction and better revealing the bidirectional asymmetric causation, overcoming the mirroring effect.

On the Different Abilities of Cross-Sample Entropy and K-Nearest-Neighbor Cross-Unpredictability in Assessing Dynamic Cardiorespiratory and Cerebrovascular Interactions

Nonlinear markers of coupling strength are often utilized to typify cardiorespiratory and cerebrovascular regulations. The computation of these indices requires techniques describing nonlinear interactions between respiration (R) and heart period (HP) and between mean arterial pressure (MAP) and mean cerebral blood velocity (MCBv). We compared two model-free methods for the assessment of dynamic HP-R and MCBv-MAP interactions, namely the cross-sample entropy (CSampEn) and k-nearest-neighbor cross-unpredictability (KNNCUP). Comparison was carried out first over simulations generated by linear and nonlinear unidirectional causal, bidirectional linear causal, and lag-zero linear noncausal models, and then over experimental data acquired from 19 subjects at supine rest during spontaneous breathing and controlled respiration at 10, 15, and 20 breaths·minute-1 as well as from 13 subjects at supine rest and during 60° head-up tilt. Linear markers were computed for comparison. We found that: (i) over simulations, CSampEn and KNNCUP exhibit different abilities in evaluating coupling strength; (ii) KNNCUP is more reliable than CSampEn when interactions occur according to a causal structure, while performances are similar in noncausal models; (iii) in healthy subjects, KNNCUP is more powerful in characterizing cardiorespiratory and cerebrovascular variability interactions than CSampEn and linear markers. We recommend KNNCUP for quantifying cardiorespiratory and cerebrovascular coupling.

Granger causality for compressively sensed sparse signals

Compressed sensing is a scheme that allows for sparse signals to be acquired, transmitted, and stored using far fewer measurements than done by conventional means employing the Nyquist sampling theorem. Since many naturally occurring signals are sparse (in some domain), compressed sensing has rapidly seen popularity in a number of applied physics and engineering applications, particularly in designing signal and image acquisition strategies, e.g., magnetic resonance imaging, quantum state tomography, scanning tunneling microscopy, and analog to digital conversion technologies. Contemporaneously, causal inference has become an important tool for the analysis and understanding of processes and their interactions in many disciplines of science, especially those dealing with complex systems. Direct causal analysis for compressively sensed data is required to avoid the task of reconstructing the compressed data. Also, for some sparse signals, such as for sparse temporal data, it may be difficult to discover causal relations directly using available data-driven or model-free causality estimation techniques. In this work, we provide a mathematical proof that structured compressed sensing matrices, specifically circulant and Toeplitz, preserve causal relationships in the compressed signal domain, as measured by Granger causality (GC). We then verify this theorem on a number of bivariate and multivariate coupled sparse signal simulations which are compressed using these matrices. We also demonstrate a real world application of network causal connectivity estimation from sparse neural spike train recordings from rat prefrontal cortex. In addition to demonstrating the effectiveness of structured matrices for GC estimation from sparse signals, we also show a computational time advantage of the proposed strategy for causal inference from compressed signals of both sparse and regular autoregressive processes as compared to standard GC estimation from original signals.

A modified extreme event-based synchronicity measure for climate time series

Extreme event-based synchronicity is a specific measure of similarity of extreme event-like time series. It is capable to capture the nonlinear interactions between climatic extreme events. In this study, we proposed a modified extreme event-based synchronicity measure that incorporates two types of extreme events (positive and negative) simultaneously in climate anomalies to characterize the synchronization and time delays. Statistical significance of the modified extreme event synchronization measure is tested by Monte-Carlo simulations. The applications of the modified extreme event-based synchronicity measure on synthetic time series verified that it was superior to the traditional event synchronicity measure. Both synchronous and antisynchronous features between climate time series could be captured by the modified measure. It is potentially applied in investigating the interrelationship between climate extremes and climate index or constructing complex networks of climate variables. In addition, this modified extreme event-based synchronicity measure could be easily applied to other types of time series just by identifying the extreme events properly.

Global droughts connected by linkages between drought hubs

Quantifying the spatial and interconnected structure of regional to continental scale droughts is one of the unsolved global hydrology problems, which is important for understanding the looming risk of mega-scale droughts and the resulting water and food scarcity and their cascading impact on the worldwide economy. Using a Complex Network analysis, this study explores the topological characteristics of global drought events based on the self-calibrated Palmer Drought Severity Index. Event Synchronization is used to measure the strength of association between the onset of droughts at different spatial locations within the time lag of 1-3 months. The network coefficients derived from the synchronization network indicate a highly heterogeneous connectivity structure underlying global drought events. Drought hotspot regions such as Southern Europe, Northeast Brazil, Australia, and Northwest USA behave as drought hubs that synchronize regionally and with other hubs at inter-continental or even inter-hemispheric scale. This observed affinity among drought hubs is equivalent to the 'rich-club phenomenon' in Network Theory, where 'rich' nodes (here, drought hubs) are tightly interconnected to form a club, implicating the possibility of simultaneous large-scale droughts over multiple continents.

Compression complexity with ordinal patterns for robust causal inference in irregularly sampled time series

Distinguishing cause from effect is a scientific challenge resisting solutions from mathematics, statistics, information theory and computer science. Compression-Complexity Causality (CCC) is a recently proposed interventional measure of causality, inspired by Wiener–Granger’s idea. It estimates causality based on change in dynamical compression-complexity (or compressibility) of the effect variable, given the cause variable. CCC works with minimal assumptions on given data and is robust to irregular-sampling, missing-data and finite-length effects. However, it only works for one-dimensional time series. We propose an ordinal pattern symbolization scheme to encode multidimensional patterns into one-dimensional symbolic sequences, and thus introduce the Permutation CCC (PCCC). We demonstrate that PCCC retains all advantages of the original CCC and can be applied to data from multidimensional systems with potentially unobserved variables which can be reconstructed using the embedding theorem. PCCC is tested on numerical simulations and applied to paleoclimate data characterized by irregular and uncertain sampling and limited numbers of samples.

Delta synchronization of Poincaré chaos in gas discharge-semiconductor systems

We introduce a new type of chaos synchronization, specifically the delta synchronization of Poincaré chaos. The method is demonstrated for the irregular dynamics in coupled gas discharge-semiconductor systems (GDSSs). It is remarkable that the processes are not generally synchronized. Our approach entirely relies on ingredients of the Poincaré chaos, which in its own turn is a consequence of the unpredictability in Poisson stable motions. The drive and response systems are in the connection, such that the latter is processed through the electric potential of the former. The absence of generalized synchronization between these systems is indicated by utilizing the conservative auxiliary system. However, the existence of common sequences of moments for finite convergence and separation confirms the delta synchronization. This can be useful for complex dynamics generation and control in electromagnetic devices. A bifurcation diagram is constructed to separate stable stationary solutions from non-trivial oscillatory ones. Phase portraits of the drive and response systems for a specific regime are provided. The results of the sequential test application to indicate the unpredictability and the delta synchronization of chaos are demonstrated in tables. The computations of the dynamical characteristics for GDSSs are carried out by using COMSOL Multiphysics version 5.6 and MATLAB version R2021b.

Comparing Cross-Sample Entropy and K-Nearest-Neighbor Cross-Predictability Approaches for the Evaluation of Cardiorespiratory and Cerebrovascular Dynamic Interactions

Quantification of the cardiorespiratory and cerebrovascular couplings is a relevant clinical issue given that their changes are considered signs of pathological status. The inherent nonlinearity of mechanisms underlying cardiorespiratory and cerebrovascular links requires nonlinear tools for their reliable evaluation. In the present study we compare two nonlinear methods for the assessment of coupling strength between two time series, namely cross-sample entropy (CSampEn) and k-nearest-neighbor cross-predictability (KNNCP). CSampEn uses a strategy that fixes the pattern length, while KNNCP optimizes the pattern length to maximize cross-predictability. CSampEn and KNNCP were applied to the beat-to-beat series of heart period (HP) and respiration (R) during a controlled breathing protocol with the aim at assessing cardiorespiratory coupling and to the beat-to-beat series of mean cerebral blood flow (MCBF) and mean arterial pressure (MAP) during an orthostatic stressor with the aim at evaluating cerebrovascular coupling. Although both the methods have the possibility to quantify the degree of HP-R and MCBF-MAP association, they exhibited different statistical power and even diverse trends in response to the considered physiological challenges. CSampEn and KNNCP are not interchangeable and should be utilized in association more than in alternative for the quantification of the HP-R and MCBF-MAP coupling strength. Clinical Relevance - This study proves that cross-entropy and cross-predictability might lead to different conclusions about cardiorespiratory and cerebrovascular couplings.

Directed coupling in multi-brain networks underlies generalized synchrony during social exchange

Advances in social neuroscience have made neural signatures of social exchange measurable simultaneously across people. This has identified brain regions differentially active during social interaction between human dyads, but the underlying systems-level mechanisms are incompletely understood. This paper introduces dynamic causal modeling and Bayesian model comparison to assess the causal and directed connectivity between two brains in the context of hyperscanning (h-DCM). In this setting, correlated neuronal responses become the data features that have to be explained by models with and without between-brain (effective) connections. Connections between brains can be understood in the context of generalized synchrony, which explains how dynamical systems become synchronized when they are coupled to each another. Under generalized synchrony, each brain state can be predicted by the other brain or a mixture of both. Our results show that effective connectivity between brains is not a feature within dyads per se but emerges selectively during social exchange. We demonstrate a causal impact of the sender's brain activity on the receiver of information, which explains previous reports of two-brain synchrony. We discuss the implications of this work; in particular, how characterizing generalized synchrony enables the discovery of between-brain connections in any social contact, and the advantage of h-DCM in studying brain function on the subject level, dyadic level, and group level within a directed model of (between) brain function.

Monitoring the Evolution of Asynchrony between Mean Arterial Pressure and Mean Cerebral Blood Flow via Cross-Entropy Methods

Cerebrovascular control is carried out by multiple nonlinear mechanisms imposing a certain degree of coupling between mean arterial pressure (MAP) and mean cerebral blood flow (MCBF). We explored the ability of two nonlinear tools in the information domain, namely cross-approximate entropy (CApEn) and cross-sample entropy (CSampEn), to assess the degree of asynchrony between the spontaneous fluctuations of MAP and MCBF. CApEn and CSampEn were computed as a function of the translation time. The analysis was carried out in 23 subjects undergoing recordings at rest in supine position (REST) and during active standing (STAND), before and after surgical aortic valve replacement (SAVR). We found that at REST the degree of asynchrony raised, and the rate of increase in asynchrony with the translation time decreased after SAVR. These results are likely the consequence of the limited variability of MAP observed after surgery at REST, more than the consequence of a modified cerebrovascular control, given that the observed differences disappeared during STAND. CApEn and CSampEn can be utilized fruitfully in the context of the evaluation of cerebrovascular control via the noninvasive acquisition of the spontaneous MAP and MCBF variability.

A complex network approach to study the extreme precipitation patterns in a river basin

The quantification of spatial propagation of extreme precipitation events is vital in water resources planning and disaster mitigation. However, quantifying these extreme events has always been challenging as many traditional methods are insufficient to capture the nonlinear interrelationships between extreme event time series. Therefore, it is crucial to develop suitable methods for analyzing the dynamics of extreme events over a river basin with a diverse climate and complicated topography. Over the last decade, complex network analysis emerged as a powerful tool to study the intricate spatiotemporal relationship between many variables in a compact way. In this study, we employ two nonlinear concepts of event synchronization and edit distance to investigate the extreme precipitation pattern in the Ganga river basin. We use the network degree to understand the spatial synchronization pattern of extreme rainfall and identify essential sites in the river basin with respect to potential prediction skills. The study also attempts to quantify the influence of precipitation seasonality and topography on extreme events. The findings of the study reveal that (1) the network degree is decreased in the southwest to northwest direction, (2) the timing of 50th percentile precipitation within a year influences the spatial distribution of degree, (3) the timing is inversely related to elevation, and (4) the lower elevation greatly influences connectivity of the sites. The study highlights that edit distance could be a promising alternative to analyze event-like data by incorporating event time and amplitude and constructing complex networks of climate extremes.

Multivariate synchronization curve: A measure of synchronization in different multivariate signals

As a method to measure the synchronization between two different sets of signals, the multivariate synchronization index (MSI) has played an irreplaceable role in the field of frequency recognition of brain-computer interface since it was proposed. On this basis, we make a generalization of MSI by using the escort distribution to replace the original distribution. In this way, MSI can be converted from a determined value to the multivariate synchronization curve, which will vary as the parameter q of the escort distribution changes. Numerical experiments are carried out on both simulated and real-world data to confirm the effectiveness of this new method. Compared with the case of MSI (i.e., q = 1), the extended form of MSI proposed in this article can obviously capture the relationship between signals more comprehensively, implying that it is a more perfect method to describe the synchronization between them. The results reveal that this method can not only effectively extract the important information contained in different signals, but also has the potential to become a practical synchronization measurement method of multivariate signals.

Convergent cross sorting for estimating dynamic coupling

Natural systems exhibit diverse behavior generated by complex interactions between their constituent parts. To characterize these interactions, we introduce Convergent Cross Sorting (CCS), a novel algorithm based on convergent cross mapping (CCM) for estimating dynamic coupling from time series data. CCS extends CCM by using the relative ranking of distances within state-space reconstructions to improve the prior methods' performance at identifying the existence, relative strength, and directionality of coupling across a wide range of signal and noise characteristics. In particular, relative to CCM, CCS has a large performance advantage when analyzing very short time series data and data from continuous dynamical systems with synchronous behavior. This advantage allows CCS to better uncover the temporal and directional relationships within systems that undergo frequent and short-lived switches in dynamics, such as neural systems. In this paper, we validate CCS on simulated data and demonstrate its applicability to electrophysiological recordings from interacting brain regions.

Finding nonlinear system equations and complex network structures from data: A sparse optimization approach

In applications of nonlinear and complex dynamical systems, a common situation is that the system can be measured, but its structure and the detailed rules of dynamical evolution are unknown. The inverse problem is to determine the system equations and structure from time series. The principle of exploiting sparse optimization to find the equations of dynamical systems from data was first articulated in 2011 by the ASU group. The basic idea is to expand the system equations into a power series or a Fourier series of a finite number of terms and then to determine the vector of the expansion coefficients based solely on data through sparse optimization. This Tutorial presents a brief review of the recent progress in this area. Issues discussed include discovering the equations of stationary or nonstationary chaotic systems to enable the prediction of critical transition and system collapse, inferring the full topology of complex oscillator networks and social networks hosting evolutionary game dynamics, and identifying partial differential equations for spatiotemporal dynamical systems. Situations where sparse optimization works or fails are pointed out. The relation with the traditional delay-coordinate embedding method is discussed, and the recent development of a model-free, data-driven prediction framework based on machine learning is mentioned.

Identifying bidirectional total and non-linear information flow in functional corticomuscular coupling during a dorsiflexion task: a pilot study

Background

The key challenge to constructing functional corticomuscular coupling (FCMC) is to accurately identify the direction and strength of the information flow between scalp electroencephalography (EEG) and surface electromyography (SEMG). Traditional TE and TDMI methods have difficulty in identifying the information interaction for short time series as they tend to rely on long and stable data, so we propose a time-delayed maximal information coefficient (TDMIC) method. With this method, we aim to investigate the directional specificity of bidirectional total and nonlinear information flow on FCMC, and to explore the neural mechanisms underlying motor dysfunction in stroke patients.

Methods

We introduced a time-delayed parameter in the maximal information coefficient to capture the direction of information interaction between two time series. We employed the linear and non-linear system model based on short data to verify the validity of our algorithm. We then used the TDMIC method to study the characteristics of total and nonlinear information flow in FCMC during a dorsiflexion task for healthy controls and stroke patients.

Results

The simulation results showed that the TDMIC method can better detect the direction of information interaction compared with TE and TDMI methods. For healthy controls, the beta band (14–30 Hz) had higher information flow in FCMC than the gamma band (31–45 Hz). Furthermore, the beta-band total and nonlinear information flow in the descending direction (EEG to EMG) was significantly higher than that in the ascending direction (EMG to EEG), whereas in the gamma band the ascending direction had significantly higher information flow than the descending direction. Additionally, we found that the strong bidirectional information flow mainly acted on Cz, C3, CP3, P3 and CPz. Compared to controls, both the beta-and gamma-band bidirectional total and nonlinear information flows of the stroke group were significantly weaker. There is no significant difference in the direction of beta- and gamma-band information flow in stroke group.

Conclusions

The proposed method could effectively identify the information interaction between short time series. According to our experiment, the beta band mainly passes downward motor control information while the gamma band features upward sensory feedback information delivery. Our observation demonstrate that the center and contralateral sensorimotor cortex play a major role in lower limb motor control. The study further demonstrates that brain damage caused by stroke disrupts the bidirectional information interaction between cortex and effector muscles in the sensorimotor system, leading to motor dysfunction.

Time-Frequency Maximal Information Coefficient Method and its Application to Functional Corticomuscular Coupling

An important challenge in the study of functional corticomuscular coupling (FCMC) is an accurate capture of the coupling relationship between the cerebral cortex and the effector muscle. The coherence method is a linear analysis method, which has certain limitations in further revealing the nonlinear coupling between neural signals. Although mutual information (MI) and transfer entropy (TE) based on information theory can capture both linear and nonlinear correlations, the equitability of these algorithms is ignored and the nonlinear components of the correlation cannot be separated. The maximal information coefficient (MIC) is a suitable method to measure the coupling between neurophysiological signals. This study extends the MIC to the time-frequency domain, named time-frequency maximal information coefficient (TFMIC), to explore the FCMC in a specific frequency band. The effectiveness, equitability, and robustness of the algorithm on the simulation data was verified and compared with coherence, TE- and MI- based methods. Simulation results showed that the TFMIC could accurately detect the coupling for different functional relationships at low noise levels. The dorsiflexion experimental results revealed that the beta-band (14-30 Hz) significant coupling was observed at channels Cz, C4, FC4, and FCz. Additionally, the results showed that the coupling was higher in the alpha-band (8-13 Hz) and beta-band (14-30 Hz) than in the gamma-band (31-45 Hz). This might be related to a transition between sensorimotor states. Specifically, the nonlinear component of FCMC was also observed at channels Cz, C4, FC4, and FCz. This study expanded the research on nonlinear coupling components in FCMC.

The extended Granger causality analysis for Hodgkin-Huxley neuronal models

How to extract directions of information flow in dynamical systems based on empirical data remains a key challenge. The Granger causality (GC) analysis has been identified as a powerful method to achieve this capability. However, the framework of the GC theory requires that the dynamics of the investigated system can be statistically linearized; i.e., the dynamics can be effectively modeled by linear regressive processes. Under such conditions, the causal connectivity can be directly mapped to the structural connectivity that mediates physical interactions within the system. However, for nonlinear dynamical systems such as the Hodgkin-Huxley (HH) neuronal circuit, the validity of the GC analysis has yet been addressed; namely, whether the constructed causal connectivity is still identical to the synaptic connectivity between neurons remains unknown. In this work, we apply the nonlinear extension of the GC analysis, i.e., the extended GC analysis, to the voltage time series obtained by evolving the HH neuronal network. In addition, we add a certain amount of measurement or observational noise to the time series to take into account the realistic situation in data acquisition in the experiment. Our numerical results indicate that the causal connectivity obtained through the extended GC analysis is consistent with the underlying synaptic connectivity of the system. This consistency is also insensitive to dynamical regimes, e.g., a chaotic or non-chaotic regime. Since the extended GC analysis could in principle be applied to any nonlinear dynamical system as long as its attractor is low dimensional, our results may potentially be extended to the GC analysis in other settings.

Implementation of two causal methods based on predictions in reconstructed state spaces

If deterministic dynamics is dominant in the data, then methods based on predictions in reconstructed state spaces can serve to detect causal relationships between and within the systems. Here we introduce two algorithms for such causal analysis. They are designed to detect causality from two time series but are potentially also applicable in a multivariate context. The first method is based on cross-predictions, and the second one on the so-called mixed predictions. In terms of performance, the cross-prediction method is considerably faster and less prone to false negatives. The predictability improvement method is slower, but in addition to causal detection, in a multivariate scenario, it also reveals which specific observables can help the most if we want to improve prediction. The study also highlights cases where our methods and state-space approaches generally seem to lose reliability. We propose a new perspective on these situations, namely that the variables under investigation have weak observability due to the complex nonlinear information flow in the system. Thus, in such cases, the failure of causality detection cannot be attributed to the methods themselves but to the use of data that do not allow reliable reconstruction of the underlying dynamics.

Partial cross mapping eliminates indirect causal influences

Causality detection likely misidentifies indirect causations as direct ones, due to the effect of causation transitivity. Although several methods in traditional frameworks have been proposed to avoid such misinterpretations, there still is a lack of feasible methods for identifying direct causations from indirect ones in the challenging situation where the variables of the underlying dynamical system are non-separable and weakly or moderately interacting. Here, we solve this problem by developing a data-based, model-independent method of partial cross mapping based on an articulated integration of three tools from nonlinear dynamics and statistics: phase-space reconstruction, mutual cross mapping, and partial correlation. We demonstrate our method by using data from different representative models and real-world systems. As direct causations are keys to the fundamental underpinnings of a variety of complex dynamics, we anticipate our method to be indispensable in unlocking and deciphering the inner mechanisms of real systems in diverse disciplines from data.

It is crucial yet challenging to identify cause-consequence relation in complex dynamical systems where direct causal links can mix with indirect ones. Leng et al. propose a data-driven model-independent method to distinguish direct from indirect causality and test its applicability to real-world data.

Evaluation of Granger Causality Measures for Constructing Networks from Multivariate Time Series

Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. These measures are in the time domain, such as model-based and information measures, the frequency domain, and the phase domain. The study aims also to compare bivariate and multivariate measures, linear and nonlinear measures, as well as the use of dimension reduction in linear model-based measures and information measures. The latter is particular relevant in the study of high-dimensional time series. For the performance of the multivariate causality measures, low and high dimensional coupled dynamical systems are considered in discrete and continuous time, as well as deterministic and stochastic. The measures are evaluated and ranked according to their ability to provide causality networks that match the original coupling structure. The simulation study concludes that the Granger causality measures using dimension reduction are superior and should be preferred particularly in studies involving many observed variables, such as multi-channel electroencephalograms and financial markets.

Noise-Induced Improvement of the Parkinsonian State: A Computational Study

The benefit of noise in improving the basal ganglia (BG) dysfunctions, especially Parkinsonian state, is explored in this paper. High frequency (≥ 100 Hz) deep brain stimulation (DBS), as a clinical effective stimulation method, has compelling and fantastic results in alleviating the motor symptoms of Parkinson's disease (PD). However, the mechanism of DBS is still unclear. And the selection of the DBS waveform parameters faces great challenges to further optimize the stimulation effects and to reduce its energy expenditure. Considering that the desynchronization of the BG neuronal activities is benefited from the forced high frequency regular spikes driven by standard high frequency DBS, we expect to explore a novel stimulation method that has capability of restoring the BG physiological firing patterns without introducing artificial high-frequency fires. In this paper, a colored noise stimulation is used as a neuromodulation method to disrupt the firing patterns of the pathological neuronal activities. A computational model of the BG that exhibits the intrinsic properties of the BG neurons and their interactions with the thalamic (Th) cells is employed. Based on the model, we investigate the effects of noise stimulation and explore the impacts of the noise stimulation parameters on both relay reliability of the Th neurons and energy expenditure of the stimulation. By comparison, it can be found that noise stimulation does not entrain the network to an artificial high-frequency firing state, but induces the pathological increased synchronous activities back to a normal physiological level. Moreover, besides the capability of restoring the neuronal state, the benefits of the noise also include its balanced waveform to avert potential tissue or electrode damage and its ability to reduce the energy expenditure to 50% less than that of the standard DBS, when the noise stimulation has low frequency (≤ 100 Hz) and appropriate intensity. Thus, the exploration of the optimal noise-induced improvement of the BG dysfunction is of great significance in treating symptoms of neurological disorders such as PD.

Causality detection in cortical seizure dynamics using cross-dynamical delay differential analysis

Most natural systems, including the brain, are highly nonlinear and complex, and determining information flow among the components that make up these dynamic systems is challenging. One such example is identifying abnormal causal interactions among different brain areas that give rise to epileptic activities. Here, we introduce cross-dynamical delay differential analysis, an extension of delay differential analysis, as a tool to establish causal relationships from time series signals. Our method can infer causality from short time series signals as well as in the presence of noise. Furthermore, we can determine the onset of generalized synchronization directly from time series data, without having to consult the underlying equations. We first validate our method on simulated datasets from coupled dynamical systems and apply the method to intracranial electroencephalography data obtained from epilepsy patients to better characterize large-scale information flow during epilepsy.

Reconstructing directional causal networks with random forest: Causality meeting machine learning

Inspired by the decision tree algorithm in machine learning, a novel causal network reconstruction framework is proposed with the name Importance Causal Analysis (ICA). The ICA framework is designed in a network level and fills the gap between traditional mutual causality detection methods and the reconstruction of causal networks. The potential of the method to identify the true causal relations in complex networks is validated by both benchmark systems and real-world data sets.

Inferring directed networks using a rank-based connectivity measure

Inferring the topology of a network using the knowledge of the signals of each of the interacting units is key to understanding real-world systems. One way to address this problem is using data-driven methods like cross-correlation or mutual information. However, these measures lack the ability to distinguish the direction of coupling. Here, we use a rank-based nonlinear interdependence measure originally developed for pairs of signals. This measure not only allows one to measure the strength but also the direction of the coupling. Our results for a system of coupled Lorenz dynamics show that we are able to consistently infer the underlying network for a subrange of the coupling strength and link density. Furthermore, we report that the addition of dynamical noise can benefit the reconstruction. Finally, we show an application to multichannel electroencephalographic recordings from an epilepsy patient.

Improving Entropy Estimates of Complex Network Topology for the Characterization of Coupling in Dynamical Systems

A new measure for the characterization of interconnected dynamical systems coupling is proposed. The method is based on the representation of time series as weighted cross-visibility networks. The weights are introduced as the metric distance between connected nodes. The structure of the networks, depending on the coupling strength, is quantified via the entropy of the weighted adjacency matrix. The method has been tested on several coupled model systems with different individual properties. The results show that the proposed measure is able to distinguish the degree of coupling of the studied dynamical systems. The original use of the geodesic distance on Gaussian manifolds as a metric distance, which is able to take into account the noise inherently superimposed on the experimental data, provides significantly better results in the calculation of the entropy, improving the reliability of the coupling estimates. The application to the interaction between the El Niño Southern Oscillation (ENSO) and the Indian Ocean Dipole and to the influence of ENSO on influenza pandemic occurrence illustrates the potential of the method for real-life problems.

Symbolic time series analysis of fNIRS signals in brain development assessment

OBJECTIVE

Assessing an infant's brain development remains a challenge for neuroscientists and pediatricians despite great technological advances. As a non-invasive neuroimaging tool, functional near-infrared spectroscopy (fNIRS) has great advantages in monitoring an infant's brain activity. To explore the dynamic features of hemodynamic changes in infants, in-pattern exponent (IPE), anti-pattern exponent (APE), as well as permutation cross-mutual information (PCMI) based on symbolic dynamics are proposed to measure the phase differences and coupling strength in oxyhemoglobin (HbO) and deoxyhemoglobin (Hb) signals from fNIRS.

APPROACH

First, simulated sinusoidal oscillation signals and four coupled nonlinear systems were employed for performance assessments. Hilbert transform based measurements of hemoglobin phase oxygenation and deoxygenation (hPod) and phase-locking index of hPod (hPodL) were calculated for comparison. Then, the IPE, APE and PCMI indices from resting state fNIRS data of preterm, term infants and adults were calculated to estimate the phase difference and coupling of HbO and Hb. All indices' performance was assessed by the degree of monotonicity (DoM). The box plots and coefficients of variation (CV) were employed to assess the measurements and robustness in the results.

MAIN RESULTS

In the simulation analysis, IPE and APE can distinguish the phase difference of two sinusoidal oscillation signals. Both hPodL and PCMI can track the strength of two coupled nonlinear systems. Compared to hPodL, the PCMI had higher DoM indices in measuring the coupling of two nonlinear systems. In the fNIRS data analysis, similar to hPod, the IPE and APE can distinguish preterm, term infants, and adults in 0.01-0.05 Hz, 0.05-0.1 Hz, and 0.01-0.1 Hz frequency bands, respectively. PCMI more effectively distinguished the term and preterm infants than hPodL in the 0.05-0.1 Hz frequency band. As symbolic time series measures, the IPE and APE were able to detect the brain developmental changes in subjects of different ages. PCMI can assess the resting-state HbO and Hb coupling changes across different developmental ages, which may reflect the metabolic and neurovascular development.

SIGNIFICANCE

The symbolic-based methodologies are promising measures for fNIRS in estimating the brain development, especially in assessing newborns' brain developmental status.

Detecting directional couplings from multivariate flows by the joint distance distribution

The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.

Introduction to Focus Issue: Causation inference and information flow in dynamical systems: Theory and applications

Questions of causation are foundational across science and often relate further to problems of control, policy decisions, and forecasts. In nonlinear dynamics and complex systems science, causation inference and information flow are closely related concepts, whereby "information" or knowledge of certain states can be thought of as coupling influence onto the future states of other processes in a complex system. While causation inference and information flow are by now classical topics, incorporating methods from statistics and time series analysis, information theory, dynamical systems, and statistical mechanics, to name a few, there remain important advancements in continuing to strengthen the theory, and pushing the context of applications, especially with the ever-increasing abundance of data collected across many fields and systems. This Focus Issue considers different aspects of these questions, both in terms of founding theory and several topical applications.

Putting the "dynamic" back into dynamic functional connectivity

The study of fluctuations in time-resolved functional connectivity is a topic of substantial current interest. As the term "dynamic functional connectivity" implies, such fluctuations are believed to arise from dynamics in the neuronal systems generating these signals. While considerable activity currently attends to methodological and statistical issues regarding dynamic functional connectivity, less attention has been paid toward its candidate causes. Here, we review candidate scenarios for dynamic (functional) connectivity that arise in dynamical systems with two or more subsystems; generalized synchronization, itinerancy (a form of metastability), and multistability. Each of these scenarios arises under different configurations of local dynamics and intersystem coupling: We show how they generate time series data with nonlinear and/or nonstationary multivariate statistics. The key issue is that time series generated by coupled nonlinear systems contain a richer temporal structure than matched multivariate (linear) stochastic processes. In turn, this temporal structure yields many of the phenomena proposed as important to large-scale communication and computation in the brain, such as phase-amplitude coupling, complexity, and flexibility. The code for simulating these dynamics is available in a freeware software platform, the Brain Dynamics Toolbox.

Comparison of six methods for the detection of causality in a bivariate time series

In this comparative study, six causality detection methods were compared, namely, the Granger vector autoregressive test, the extended Granger test, the kernel version of the Granger test, the conditional mutual information (transfer entropy), the evaluation of cross mappings between state spaces, and an assessment of predictability improvement due to the use of mixed predictions. Seven test data sets were analyzed: linear coupling of autoregressive models, a unidirectional connection of two Hénon systems, a unidirectional connection of chaotic systems of Rössler and Lorenz type and of two different Rössler systems, an example of bidirectionally connected two-species systems, a fishery model as an example of two correlated observables without a causal relationship, and an example of mediated causality. We tested not only 20000 points long clean time series but also noisy and short variants of the data. The standard and the extended Granger tests worked only for the autoregressive models. The remaining methods were more successful with the more complex test examples, although they differed considerably in their capability to reveal the presence and the direction of coupling and to distinguish causality from mere correlation.

Robustness and versatility of a nonlinear interdependence method for directional coupling detection from spike trains

The detection of directional couplings between dynamics based on measured spike trains is a crucial problem in the understanding of many different systems. In particular, in neuroscience it is important to assess the connectivity between neurons. One of the approaches that can estimate directional coupling from the analysis of point processes is the nonlinear interdependence measure L. Although its efficacy has already been demonstrated, it still needs to be tested under more challenging and realistic conditions prior to an application to real data. Thus, in this paper we use the Hindmarsh-Rose model system to test the method in the presence of noise and for different spiking regimes. We also examine the influence of different parameters and spike train distances. Our results show that the measure L is versatile and robust to various types of noise, and thus suitable for application to experimental data.

Detection of time delays and directional interactions based on time series from complex dynamical systems

Data-based and model-free accurate identification of intrinsic time delays and directional interactions is an extremely challenging problem in complex dynamical systems and their networks reconstruction. A model-free method with new scores is proposed to be generally capable of detecting single, multiple, and distributed time delays. The method is applicable not only to mutually interacting dynamical variables but also to self-interacting variables in a time-delayed feedback loop. Validation of the method is carried out using physical, biological, and ecological models and real data sets. Especially, applying the method to air pollution data and hospital admission records of cardiovascular diseases in Hong Kong reveals the major air pollutants as a cause of the diseases and, more importantly, it uncovers a hidden time delay (about 30-40 days) in the causal influence that previous studies failed to detect. The proposed method is expected to be universally applicable to ascertaining and quantifying subtle interactions (e.g., causation) in complex systems arising from a broad range of disciplines.

Coupling strength versus coupling impact in nonidentical bidirectionally coupled dynamics

The understanding of interacting dynamics is important for the characterization of real-world networks. In general, real-world networks are heterogeneous in the sense that each node of the network is a dynamics with different properties. For coupled nonidentical dynamics symmetric interactions are not straightforwardly defined from the coupling strength values. Thus, a challenging issue is whether we can define a symmetric interaction in this asymmetric setting. To address this problem we introduce the notion of the coupling impact. The coupling impact considers not only the coupling strength but also the energy of the individual dynamics, which is conveyed via the coupling. To illustrate this concept, we follow a data-driven approach by analyzing signals from pairs of coupled model dynamics using two different connectivity measures. We find that the coupling impact, but not the coupling strength, correctly detects a symmetric interaction between pairs of coupled dynamics regardless of their degree of asymmetry. Therefore, this approach allows us to reveal the real impact that one dynamics has on the other and hence to define symmetric interactions in pairs of nonidentical dynamics.

Testing for causality in reconstructed state spaces by an optimized mixed prediction method

In this study, a method of causality detection was designed to reveal coupling between dynamical systems represented by time series. The method is based on the predictions in reconstructed state spaces. The results of the proposed method were compared with outcomes of two other methods, the Granger VAR test of causality and the convergent cross-mapping. We used two types of test data. The first test example is a unidirectional connection of chaotic systems of Rössler and Lorenz type. The second one, the fishery model, is an example of two correlated observables without a causal relationship. The results showed that the proposed method of optimized mixed prediction was able to reveal the presence and the direction of coupling and distinguish causality from mere correlation as well.

Computing algebraic transfer entropy and coupling directions via transcripts

Most random processes studied in nonlinear time series analysis take values on sets endowed with a group structure, e.g., the real and rational numbers, and the integers. This fact allows to associate with each pair of group elements a third element, called their transcript, which is defined as the product of the second element in the pair times the first one. The transfer entropy of two such processes is called algebraic transfer entropy. It measures the information transferred between two coupled processes whose values belong to a group. In this paper, we show that, subject to one constraint, the algebraic transfer entropy matches the (in general, conditional) mutual information of certain transcripts with one variable less. This property has interesting practical applications, especially to the analysis of short time series. We also derive weak conditions for the 3-dimensional algebraic transfer entropy to yield the same coupling direction as the corresponding mutual information of transcripts. A related issue concerns the use of mutual information of transcripts to determine coupling directions in cases where the conditions just mentioned are not fulfilled. We checked the latter possibility in the lowest dimensional case with numerical simulations and cardiovascular data, and obtained positive results.

Detecting Causality by Combined Use of Multiple Methods: Climate and Brain Examples

Identifying causal relations from time series is the first step to understanding the behavior of complex systems. Although many methods have been proposed, few papers have applied multiple methods together to detect causal relations based on time series generated from coupled nonlinear systems with some unobserved parts. Here we propose the combined use of three methods and a majority vote to infer causality under such circumstances. Two of these methods are proposed here for the first time, and all of the three methods can be applied even if the underlying dynamics is nonlinear and there are hidden common causes. We test our methods with coupled logistic maps, coupled Rössler models, and coupled Lorenz models. In addition, we show from ice core data how the causal relations among the temperature, the CH4 level, and the CO2 level in the atmosphere changed in the last 800,000 years, a conclusion also supported by irregularly sampled data analysis. Moreover, these methods show how three regions of the brain interact with each other during the visually cued, two-choice arm reaching task. Especially, we demonstrate that this is due to bottom up influences at the beginning of the task, while there exist mutual influences between the posterior medial prefrontal cortex and the presupplementary motor area. Based on our results, we conclude that identifying causality with an appropriate ensemble of multiple methods ensures the validity of the obtained results more firmly.

A Nonlinear Causality Estimator Based on Non-Parametric Multiplicative Regression

Causal prediction has become a popular tool for neuroscience applications, as it allows the study of relationships between different brain areas during rest, cognitive tasks or brain disorders. We propose a nonparametric approach for the estimation of nonlinear causal prediction for multivariate time series. In the proposed estimator, C NPMR , Autoregressive modeling is replaced by Nonparametric Multiplicative Regression (NPMR). NPMR quantifies interactions between a response variable (effect) and a set of predictor variables (cause); here, we modified NPMR for model prediction. We also demonstrate how a particular measure, the sensitivity Q, could be used to reveal the structure of the underlying causal relationships. We apply C NPMR on artificial data with known ground truth (5 datasets), as well as physiological data (2 datasets). C NPMR correctly identifies both linear and nonlinear causal connections that are present in the artificial data, as well as physiologically relevant connectivity in the real data, and does not seem to be affected by filtering. The Sensitivity measure also provides useful information about the latent connectivity.The proposed estimator addresses many of the limitations of linear Granger causality and other nonlinear causality estimators. C NPMR is compared with pairwise and conditional Granger causality (linear) and Kernel-Granger causality (nonlinear). The proposed estimator can be applied to pairwise or multivariate estimations without any modifications to the main method. Its nonpametric nature, its ability to capture nonlinear relationships and its robustness to filtering make it appealing for a number of applications.

Multi-Dimensional Dynamics of Human Electromagnetic Brain Activity

Magnetoencephalography (MEG) and electroencephalography (EEG) are invaluable neuroscientific tools for unveiling human neural dynamics in three dimensions (space, time, and frequency), which are associated with a wide variety of perceptions, cognition, and actions. MEG/EEG also provides different categories of neuronal indices including activity magnitude, connectivity, and network properties along the three dimensions. In the last 20 years, interest has increased in inter-regional connectivity and complex network properties assessed by various sophisticated scientific analyses. We herein review the definition, computation, short history, and pros and cons of connectivity and complex network (graph-theory) analyses applied to MEG/EEG signals. We briefly describe recent developments in source reconstruction algorithms essential for source-space connectivity and network analyses. Furthermore, we discuss a relatively novel approach used in MEG/EEG studies to examine the complex dynamics represented by human brain activity. The correct and effective use of these neuronal metrics provides a new insight into the multi-dimensional dynamics of the neural representations of various functions in the complex human brain.

Detection of Coupling in Short Physiological Series by a Joint Distribution Entropy Method

OBJECTIVE

In this study, we developed a joint distribution entropy (JDistEn) method to robustly estimate the coupling in short physiological series.

METHODS

The JDistEn method is derived from a joint distance matrix which is constructed from a combination of the distance matrix corresponding to each individual data channel using a geometric mean calculation. A coupled Rössler system and a coupled dual-kinetics neural mass model were used to examine how well JDistEn performed, specifically, its sensitivity for detecting weak coupling, its consistency in gauging coupling strength, and its reliability in processing input of decreased data length. Performance of JDistEn in estimating physiological coupling was further examined with bivariate electroencephalography data from rats and RR interval and diastolic time interval series from human beings. Cross-sample entropy (XSampEn), cross-conditional entropy (XCE), and Shannon entropy of diagonal lines in the joint recurrence plots (JENT) were applied for purposes of comparison.

RESULTS

Simulation results suggest that JDistEn showed markedly higher sensitivity than XSampEn, XCE, and JENT for dynamics in weak coupling, although as the simulation models were more intensively coupled, JDistEn performance was comparable to the three others. In addition, this improved sensitivity was much more pronounced for short datasets. Experimental results further confirmed that JDistEn outperformed XSampEn, XCE, and JENT for detecting weak coupling, especially for short physiological data.

CONCLUSION

This study suggested that our proposed JDistEn could be useful for continuous and even real-time coupling analysis for physiological signals in clinical practice.

Distinguishing time-delayed causal interactions using convergent cross mapping

An important problem across many scientific fields is the identification of causal effects from observational data alone. Recent methods (convergent cross mapping, CCM) have made substantial progress on this problem by applying the idea of nonlinear attractor reconstruction to time series data. Here, we expand upon the technique of CCM by explicitly considering time lags. Applying this extended method to representative examples (model simulations, a laboratory predator-prey experiment, temperature and greenhouse gas reconstructions from the Vostok ice core, and long-term ecological time series collected in the Southern California Bight), we demonstrate the ability to identify different time-delayed interactions, distinguish between synchrony induced by strong unidirectional-forcing and true bidirectional causality, and resolve transitive causal chains.

Relating Granger causality to long-term causal effects

In estimation of causal couplings between observed processes, it is important to characterize coupling roles at various time scales. The widely used Granger causality reflects short-term effects: it shows how strongly perturbations of a current state of one process affect near future states of another process, and it quantifies that via prediction improvement (PI) in autoregressive models. However, it is often more important to evaluate the effects of coupling on long-term statistics, e.g., to find out how strongly the presence of coupling changes the variance of a driven process as compared to an uncoupled case. No general relationships between Granger causality and such long-term effects are known. Here, we pose the problem of relating these two types of coupling characteristics, and we solve it for a class of stochastic systems. Namely, for overdamped linear oscillators, we rigorously derive that the above long-term effect is proportional to the short-term effects, with the proportionality coefficient depending on the prediction interval and relaxation times. We reveal that this coefficient is typically considerably greater than unity so that small normalized PI values may well correspond to quite large long-term effects of coupling. The applicability of the derived relationship to wider classes of systems, its limitations, and its value for further research are discussed. To give a real-world example, we analyze couplings between large-scale climatic processes related to sea surface temperature variations in equatorial Pacific and North Atlantic regions.

Online Epileptic Seizure Prediction Using Wavelet-Based Bi-Phase Correlation of Electrical Signals Tomography

Considerable efforts have been made in order to predict seizures. Among these methods, the ones that quantify synchronization between brain areas, are the most important methods. However, to date, a practically acceptable result has not been reported. In this paper, we use a synchronization measurement method that is derived according to the ability of bi-spectrum in determining the nonlinear properties of a system. In this method, first, temporal variation of the bi-spectrum of different channels of electro cardiography (ECoG) signals are obtained via an extended wavelet-based time-frequency analysis method; then, to compare different channels, the bi-phase correlation measure is introduced. Since, in this way, the temporal variation of the amount of nonlinear coupling between brain regions, which have not been considered yet, are taken into account, results are more reliable than the conventional phase-synchronization measures. It is shown that, for 21 patients of FSPEEG database, bi-phase correlation can discriminate the pre-ictal and ictal states, with very low false positive rates (FPRs) (average: 0.078/h) and high sensitivity (100%). However, the proposed seizure predictor still cannot significantly overcome the random predictor for all patients.