Nonuniform state-space reconstruction and coupling detection

Detecting Abnormal Pattern of Epileptic Seizures via Temporal Synchronization of EEG Signals

OBJECTIVE

Synchronization phenomena of epileptic electroencephalography (EEG) have long been studied. In this study, we aim at investigating the spatial-temporal synchronization pattern in epileptic human brains using the spectral graph theoretic features extracted from scalp EEG and developing an efficient multivariate approach for detecting seizure onsets in real time.

METHODS

A complex network model is used for representing the recurrence pattern of EEG signals, based on which the temporal synchronization patterns are quantified using the spectral graph theoretic features. Furthermore, a statistical control chart is applied to the extracted features overtime for monitoring the transits from normal to epileptic states in multivariate EEG systems.

RESULTS

Our method is tested on 23 patients from CHB-MIT Scalp EEG database. The results show that the graph theoretic feature yields a high sensitivity (  ∼ 98%) and low latency (  ∼ 6 s) on average, and seizure onsets in 18 patients are 100% detected.

CONCLUSION

Our approach validates the increased temporal synchronization in epileptic EEG and achieves a comparable detection performance to previous studies.

SIGNIFICANCE

We characterize the temporal synchronization patterns of epileptic EEG using spectral network metrics. In addition, we found significant changes in temporal synchronization in epileptic EEG, which enable a patient-specific approach for real-time seizure detection for personalized diagnosis and treatment.

Inference of topology and the nature of synapses, and the flow of information in neuronal networks

The characterization of neuronal connectivity is one of the most important matters in neuroscience. In this work, we show that a recently proposed informational quantity, the causal mutual information, employed with an appropriate methodology, can be used not only to correctly infer the direction of the underlying physical synapses, but also to identify their excitatory or inhibitory nature, considering easy to handle and measure bivariate time series. The success of our approach relies on a surprising property found in neuronal networks by which nonadjacent neurons do "understand" each other (positive mutual information), however, this exchange of information is not capable of causing effect (zero transfer entropy). Remarkably, inhibitory connections, responsible for enhancing synchronization, transfer more information than excitatory connections, known to enhance entropy in the network. We also demonstrate that our methodology can be used to correctly infer directionality of synapses even in the presence of dynamic and observational Gaussian noise, and is also successful in providing the effective directionality of intermodular connectivity, when only mean fields can be measured.

Low-dimensional approximation searching strategy for transfer entropy from non-uniform embedding

Transfer entropy from non-uniform embedding is a popular tool for the inference of causal relationships among dynamical subsystems. In this study we present an approach that makes use of low-dimensional conditional mutual information quantities to decompose the original high-dimensional conditional mutual information in the searching procedure of non-uniform embedding for significant variables at different lags. We perform a series of simulation experiments to assess the sensitivity and specificity of our proposed method to demonstrate its advantage compared to previous algorithms. The results provide concrete evidence that low-dimensional approximations can help to improve the statistical accuracy of transfer entropy in multivariate causality analysis and yield a better performance over other methods. The proposed method is especially efficient as the data length grows.

Association between autonomic control indexes and mortality in subjects admitted to intensive care unit

This study checks whether autonomic markers derived from spontaneous fluctuations of heart period (HP) and systolic arterial pressure (SAP) and from their interactions with spontaneous or mechanical respiration (R) are associated with mortality in patients admitted to intensive care unit (ICU). Three-hundred consecutive HP, SAP and R values were recorded during the first day in ICU in 123 patients. Population was divided into survivors (SURVs, n = 83) and non-survivors (NonSURVs, n = 40) according to the outcome. SURVs and NonSURVs were aged- and gender-matched. All subjects underwent modified head-up tilt (MHUT) by tilting the bed back rest segment to 60°. Autonomic control indexes were computed using time-domain, spectral, cross-spectral, complexity, symbolic and causality techniques via univariate, bivariate and conditional approaches. SAP indexes derived from time-domain, model-free complexity and symbolic approaches were associated with the endpoint, while none of HP variability markers was. The association was more powerful during MHUT. Linear cross-spectral and causality indexes were useless to separate SURVs from NonSURVs, while nonlinear bivariate symbolic markers were successful. When indexes were combined with clinical scores, only SAP variance provided complementary information. Cardiovascular control variability indexes, especially when derived after an autonomic challenge such as MHUT, can improve mortality risk stratification in ICU.

Directionality indices: Testing information transfer with surrogate correction

Directionality indices can be used as an indicator of the asymmetry in coupling between systems and have found particular application in relation to neurological systems. The directionality index between two systems is a function of measures of information transfer in both directions. Here we illustrate that before inferring the directionality of coupling it is first necessary to consider the use of appropriate tests of significance. We propose a surrogate corrected directionality index which incorporates such testing. We also highlight the differences between testing the significance of the directionality index itself versus testing the individual measures of information transfer in each direction. To validate the approach we compared two different methods of estimating coupling, both of which have previously been used to estimate directionality indices. These were the modeling-based evolution map approach and a conditional mutual information (CMI) method for calculating dynamic information rates. For the CMI-based approach we also compared two different methods for estimating the CMI, an equiquantization-based estimator and a k-nearest neighbors estimator.

Towards the identification of subjects prone to develop atrial fibrillation after coronary artery bypass graft surgery via univariate and multivariate complexity analysis of heart period variability

The assessment of cardiovascular control complexity as derived from spontaneous heart period (HP) fluctuations can be improved by exploiting a multivariate (MV) approach. This work proposes the assessment of a normalized complexity index (NCI) of HP variability according to a k-nearest-neighbor approach based on local predictability performed in a MV nonuniform embedding space. The method allows the selection of the past components of HP, systolic arterial pressure (SAP) and respiration (R) most useful for the prediction of HP fluctuations. The NCI derived from the MV approach (NCIMV) was compared to a NCI computed via the same technique applied in a univariate (UV) embedding space (NCIUV) formed exclusively by HP past samples. Indexes were computed in 130 patients undergoing coronary artery bypass graft (CABG) surgery before and after the induction of general anesthesia. Thirty-eight subjects developed atrial fibrillation (AF) after surgery, while the remaining ones did not (noAF, n=92). Both NCIUV and NCIMV could separate AF from noAF patients and revealed a larger complexity of the AF subjects. However, the statistical power of the NCIMV was superior given that the probability of type I error was smaller than that of NCIUV. The assessment of cardiac control complexity could improve risk stratification of patients at risk of developing AF after CABG surgery.

Detection of coupling delay: A problem not yet solved

Nonparametric detection of coupling delay in unidirectionally and bidirectionally coupled nonlinear dynamical systems is examined. Both continuous and discrete-time systems are considered. Two methods of detection are assessed-the method based on conditional mutual information-the CMI method (also known as the transfer entropy method) and the method of convergent cross mapping-the CCM method. Computer simulations show that neither method is generally reliable in the detection of coupling delays. For continuous-time chaotic systems, the CMI method appears to be more sensitive and applicable in a broader range of coupling parameters than the CCM method. In the case of tested discrete-time dynamical systems, the CCM method has been found to be more sensitive, while the CMI method required much stronger coupling strength in order to bring correct results. However, when studied systems contain a strong oscillatory component in their dynamics, results of both methods become ambiguous. The presented study suggests that results of the tested algorithms should be interpreted with utmost care and the nonparametric detection of coupling delay, in general, is a problem not yet solved.

Assessment of resampling methods for causality testing: A note on the US inflation behavior

Different resampling methods for the null hypothesis of no Granger causality are assessed in the setting of multivariate time series, taking into account that the driving-response coupling is conditioned on the other observed variables. As appropriate test statistic for this setting, the partial transfer entropy (PTE), an information and model-free measure, is used. Two resampling techniques, time-shifted surrogates and the stationary bootstrap, are combined with three independence settings (giving a total of six resampling methods), all approximating the null hypothesis of no Granger causality. In these three settings, the level of dependence is changed, while the conditioning variables remain intact. The empirical null distribution of the PTE, as the surrogate and bootstrapped time series become more independent, is examined along with the size and power of the respective tests. Additionally, we consider a seventh resampling method by contemporaneously resampling the driving and the response time series using the stationary bootstrap. Although this case does not comply with the no causality hypothesis, one can obtain an accurate sampling distribution for the mean of the test statistic since its value is zero under H0. Results indicate that as the resampling setting gets more independent, the test becomes more conservative. Finally, we conclude with a real application. More specifically, we investigate the causal links among the growth rates for the US CPI, money supply and crude oil. Based on the PTE and the seven resampling methods, we consistently find that changes in crude oil cause inflation conditioning on money supply in the post-1986 period. However this relationship cannot be explained on the basis of traditional cost-push mechanisms.

Atypical transistor-based chaotic oscillators: Design, realization, and diversity

In this paper, we show that novel autonomous chaotic oscillators based on one or two bipolar junction transistors and a limited number of passive components can be obtained via random search with suitable heuristics. Chaos is a pervasive occurrence in these circuits, particularly after manual adjustment of a variable resistor placed in series with the supply voltage source. Following this approach, 49 unique circuits generating chaotic signals when physically realized were designed, representing the largest collection of circuits of this kind to date. These circuits are atypical as they do not trivially map onto known topologies or variations thereof. They feature diverse spectra and predominantly anti-persistent monofractal dynamics. Notably, we recurrently found a circuit comprising one resistor, one transistor, two inductors, and one capacitor, which generates a range of attractors depending on the parameter values. We also found a circuit yielding an irregular quantized spike-train resembling some aspects of neural discharge and another one generating a double-scroll attractor, which represent the smallest known transistor-based embodiments of these behaviors. Through three representative examples, we additionally show that diffusive coupling of heterogeneous oscillators of this kind may give rise to complex entrainment, such as lag synchronization with directed information transfer and generalized synchronization. The replicability and reproducibility of the experimental findings are good.

Coupling strength versus coupling impact in nonidentical bidirectionally coupled dynamics

The understanding of interacting dynamics is important for the characterization of real-world networks. In general, real-world networks are heterogeneous in the sense that each node of the network is a dynamics with different properties. For coupled nonidentical dynamics symmetric interactions are not straightforwardly defined from the coupling strength values. Thus, a challenging issue is whether we can define a symmetric interaction in this asymmetric setting. To address this problem we introduce the notion of the coupling impact. The coupling impact considers not only the coupling strength but also the energy of the individual dynamics, which is conveyed via the coupling. To illustrate this concept, we follow a data-driven approach by analyzing signals from pairs of coupled model dynamics using two different connectivity measures. We find that the coupling impact, but not the coupling strength, correctly detects a symmetric interaction between pairs of coupled dynamics regardless of their degree of asymmetry. Therefore, this approach allows us to reveal the real impact that one dynamics has on the other and hence to define symmetric interactions in pairs of nonidentical dynamics.

Computing algebraic transfer entropy and coupling directions via transcripts

Most random processes studied in nonlinear time series analysis take values on sets endowed with a group structure, e.g., the real and rational numbers, and the integers. This fact allows to associate with each pair of group elements a third element, called their transcript, which is defined as the product of the second element in the pair times the first one. The transfer entropy of two such processes is called algebraic transfer entropy. It measures the information transferred between two coupled processes whose values belong to a group. In this paper, we show that, subject to one constraint, the algebraic transfer entropy matches the (in general, conditional) mutual information of certain transcripts with one variable less. This property has interesting practical applications, especially to the analysis of short time series. We also derive weak conditions for the 3-dimensional algebraic transfer entropy to yield the same coupling direction as the corresponding mutual information of transcripts. A related issue concerns the use of mutual information of transcripts to determine coupling directions in cases where the conditions just mentioned are not fulfilled. We checked the latter possibility in the lowest dimensional case with numerical simulations and cardiovascular data, and obtained positive results.

New Insights into Signed Path Coefficient Granger Causality Analysis

Granger causality analysis, as a time series analysis technique derived from econometrics, has been applied in an ever-increasing number of publications in the field of neuroscience, including fMRI, EEG/MEG, and fNIRS. The present study mainly focuses on the validity of “signed path coefficient Granger causality,” a Granger-causality-derived analysis method that has been adopted by many fMRI researches in the last few years. This method generally estimates the causality effect among the time series by an order-1 autoregression, and defines a positive or negative coefficient as an “excitatory” or “inhibitory” influence. In the current work we conducted a series of computations from resting-state fMRI data and simulation experiments to illustrate the signed path coefficient method was flawed and untenable, due to the fact that the autoregressive coefficients were not always consistent with the real causal relationships and this would inevitablely lead to erroneous conclusions. Overall our findings suggested that the applicability of this kind of causality analysis was rather limited, hence researchers should be more cautious in applying the signed path coefficient Granger causality to fMRI data to avoid misinterpretation.

Discrimination of coupling structures using causality networks from multivariate time series

Measures of Granger causality on multivariate time series have been used to form the so-called causality networks. A causality network represents the interdependence structure of the underlying dynamical system or coupled dynamical systems, and its properties are quantified by network indices. In this work, it is investigated whether network indices on networks generated by an appropriate Granger causality measure can discriminate different coupling structures. The information based Granger causality measure of partial mutual information from mixed embedding (PMIME) is used to form causality networks, and a large number of network indices are ranked according to their ability to discriminate the different coupling structures. The evaluation of the network indices is done with a simulation study based on two dynamical systems, the coupled Mackey-Glass delay differential equations and the neural mass model, both of 25 variables, and three prototypes of coupling structures, i.e., random, small-world, and scale-free. It is concluded that the setting of PMIME combined with a network index attains high level of discrimination of the coupling structures solely on the basis of the observed multivariate time series. This approach is demonstrated to identify epileptic seizures emerging during electroencephalogram recordings.

Multiscale modeling of brain dynamics: from single neurons and networks to mathematical tools

The extreme complexity of the brain naturally requires mathematical modeling approaches on a large variety of scales; the spectrum ranges from single neuron dynamics over the behavior of groups of neurons to neuronal network activity. Thus, the connection between the microscopic scale (single neuron activity) to macroscopic behavior (emergent behavior of the collective dynamics) and vice versa is a key to understand the brain in its complexity. In this work, we attempt a review of a wide range of approaches, ranging from the modeling of single neuron dynamics to machine learning. The models include biophysical as well as data-driven phenomenological models. The discussed models include Hodgkin-Huxley, FitzHugh-Nagumo, coupled oscillators (Kuramoto oscillators, Rössler oscillators, and the Hindmarsh-Rose neuron), Integrate and Fire, networks of neurons, and neural field equations. In addition to the mathematical models, important mathematical methods in multiscale modeling and reconstruction of the causal connectivity are sketched. The methods include linear and nonlinear tools from statistics, data analysis, and time series analysis up to differential equations, dynamical systems, and bifurcation theory, including Granger causal connectivity analysis, phase synchronization connectivity analysis, principal component analysis (PCA), independent component analysis (ICA), and manifold learning algorithms such as ISOMAP, and diffusion maps and equation-free techniques. WIREs Syst Biol Med 2016, 8:438-458. doi: 10.1002/wsbm.1348 For further resources related to this article, please visit the WIREs website.

Causality networks from multivariate time series and application to epilepsy

Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. For this, realizations on high dimensional coupled dynamical systems are considered and the performance of the Granger causality measures is evaluated, seeking for the measures that form networks closest to the true network of the dynamical system. In particular, the comparison focuses on Granger causality measures that reduce the state space dimension when many variables are observed. Further, the linear and nonlinear Granger causality measures of dimension reduction are compared to a standard Granger causality measure on electroencephalographic (EEG) recordings containing episodes of epileptiform discharges.

Parsimonious description for predicting high-dimensional dynamics

When we observe a system, we often cannot observe all its variables and may have some of its limited measurements. Under such a circumstance, delay coordinates, vectors made of successive measurements, are useful to reconstruct the states of the whole system. Although the method of delay coordinates is theoretically supported for high-dimensional dynamical systems, practically there is a limitation because the calculation for higher-dimensional delay coordinates becomes more expensive. Here, we propose a parsimonious description of virtually infinite-dimensional delay coordinates by evaluating their distances with exponentially decaying weights. This description enables us to predict the future values of the measurements faster because we can reuse the calculated distances, and more accurately because the description naturally reduces the bias of the classical delay coordinates toward the stable directions. We demonstrate the proposed method with toy models of the atmosphere and real datasets related to renewable energy.

Low-dimensional attractor for neural activity from local field potentials in optogenetic mice

We used optogenetic mice to investigate possible nonlinear responses of the medial prefrontal cortex (mPFC) local network to light stimuli delivered by a 473 nm laser through a fiber optics. Every 2 s, a brief 10 ms light pulse was applied and the local field potentials (LFPs) were recorded with a 10 kHz sampling rate. The experiment was repeated 100 times and we only retained and analyzed data from six animals that showed stable and repeatable response to optical stimulations. The presence of nonlinearity in our data was checked using the null hypothesis that the data were linearly correlated in the temporal domain, but were random otherwise. For each trail, 100 surrogate data sets were generated and both time reversal asymmetry and false nearest neighbor (FNN) were used as discriminating statistics for the null hypothesis. We found that nonlinearity is present in all LFP data. The first 0.5 s of each 2 s LFP recording were dominated by the transient response of the networks. For each trial, we used the last 1.5 s of steady activity to measure the phase resetting induced by the brief 10 ms light stimulus. After correcting the LFPs for the effect of phase resetting, additional preprocessing was carried out using dendrograms to identify “similar” groups among LFP trials. We found that the steady dynamics of mPFC in response to light stimuli could be reconstructed in a three-dimensional phase space with topologically similar “8”-shaped attractors across different animals. Our results also open the possibility of designing a low-dimensional model for optical stimulation of the mPFC local network.

Relating Granger causality to long-term causal effects

In estimation of causal couplings between observed processes, it is important to characterize coupling roles at various time scales. The widely used Granger causality reflects short-term effects: it shows how strongly perturbations of a current state of one process affect near future states of another process, and it quantifies that via prediction improvement (PI) in autoregressive models. However, it is often more important to evaluate the effects of coupling on long-term statistics, e.g., to find out how strongly the presence of coupling changes the variance of a driven process as compared to an uncoupled case. No general relationships between Granger causality and such long-term effects are known. Here, we pose the problem of relating these two types of coupling characteristics, and we solve it for a class of stochastic systems. Namely, for overdamped linear oscillators, we rigorously derive that the above long-term effect is proportional to the short-term effects, with the proportionality coefficient depending on the prediction interval and relaxation times. We reveal that this coefficient is typically considerably greater than unity so that small normalized PI values may well correspond to quite large long-term effects of coupling. The applicability of the derived relationship to wider classes of systems, its limitations, and its value for further research are discussed. To give a real-world example, we analyze couplings between large-scale climatic processes related to sea surface temperature variations in equatorial Pacific and North Atlantic regions.

Neural networks with non-uniform embedding and explicit validation phase to assess Granger causality

A challenging problem when studying a dynamical system is to find the interdependencies among its individual components. Several algorithms have been proposed to detect directed dynamical influences between time series. Two of the most used approaches are a model-free one (transfer entropy) and a model-based one (Granger causality). Several pitfalls are related to the presence or absence of assumptions in modeling the relevant features of the data. We tried to overcome those pitfalls using a neural network approach in which a model is built without any a priori assumptions. In this sense this method can be seen as a bridge between model-free and model-based approaches. The experiments performed will show that the method presented in this work can detect the correct dynamical information flows occurring in a system of time series. Additionally we adopt a non-uniform embedding framework according to which only the past states that actually help the prediction are entered into the model, improving the prediction and avoiding the risk of overfitting. This method also leads to a further improvement with respect to traditional Granger causality approaches when redundant variables (i.e. variables sharing the same information about the future of the system) are involved. Neural networks are also able to recognize dynamics in data sets completely different from the ones used during the training phase.

Direct Causal Networks for the Study of Transcranial Magnetic Stimulation Effects on Focal Epileptiform Discharges

Background: Transcranial magnetic stimulation (TMS) can have inhibitory effects on epileptiform discharges (EDs) of patients with focal seizures. However, the brain connectivity before, during and after EDs, with or without the administration of TMS, has not been extensively explored. Objective: To investigate the brain network of effective connectivity during ED with and without TMS in patients with focal seizures. Methods: For the effective connectivity a direct causality measure is applied termed partial mutual information from mixed embedding (PMIME). TMS-EEG data from two patients with focal seizures were analyzed. Each EEG record contained a number of EDs in the majority of which TMS was administered over the epileptic focus. As a control condition, sham stimulation over the epileptogenic zone or real TMS at a distance from the epileptic focus was also performed. The change in brain connectivity structure was investigated from the causal networks formed at each sliding window. Conclusion: The PMIME could detect distinct changes in the network structure before, within, and after ED. The administration of real TMS over the epileptic focus, in contrast to sham stimulation, terminated the ED prematurely in a node-specific manner and regained the network structure as if it would have terminated spontaneously.

Causation entropy from symbolic representations of dynamical systems

Identification of causal structures and quantification of direct information flows in complex systems is a challenging yet important task, with practical applications in many fields. Data generated by dynamical processes or large-scale systems are often symbolized, either because of the finite resolution of the measurement apparatus, or because of the need of statistical estimation. By algorithmic application of causation entropy, we investigated the effects of symbolization on important concepts such as Markov order and causal structure of the tent map. We uncovered that these quantities depend nonmonotonically and, most of all, sensitively on the choice of symbolization. Indeed, we show that Markov order and causal structure do not necessarily converge to their original analog counterparts as the resolution of the partitioning becomes finer.

Estimating the decomposition of predictive information in multivariate systems

In the study of complex systems from observed multivariate time series, insight into the evolution of one system may be under investigation, which can be explained by the information storage of the system and the information transfer from other interacting systems. We present a framework for the model-free estimation of information storage and information transfer computed as the terms composing the predictive information about the target of a multivariate dynamical process. The approach tackles the curse of dimensionality employing a nonuniform embedding scheme that selects progressively, among the past components of the multivariate process, only those that contribute most, in terms of conditional mutual information, to the present target process. Moreover, it computes all information-theoretic quantities using a nearest-neighbor technique designed to compensate the bias due to the different dimensionality of individual entropy terms. The resulting estimators of prediction entropy, storage entropy, transfer entropy, and partial transfer entropy are tested on simulations of coupled linear stochastic and nonlinear deterministic dynamic processes, demonstrating the superiority of the proposed approach over the traditional estimators based on uniform embedding. The framework is then applied to multivariate physiologic time series, resulting in physiologically well-interpretable information decompositions of cardiovascular and cardiorespiratory interactions during head-up tilt and of joint brain-heart dynamics during sleep.

The causal inference of cortical neural networks during music improvisations

We present an EEG study of two music improvisation experiments. Professional musicians with high level of improvisation skills were asked to perform music either according to notes (composed music) or in improvisation. Each piece of music was performed in two different modes: strict mode and "let-go" mode. Synchronized EEG data was measured from both musicians and listeners. We used one of the most reliable causality measures: conditional Mutual Information from Mixed Embedding (MIME), to analyze directed correlations between different EEG channels, which was combined with network theory to construct both intra-brain and cross-brain networks. Differences were identified in intra-brain neural networks between composed music and improvisation and between strict mode and "let-go" mode. Particular brain regions such as frontal, parietal and temporal regions were found to play a key role in differentiating the brain activities between different playing conditions. By comparing the level of degree centralities in intra-brain neural networks, we found a difference between the response of musicians and the listeners when comparing the different playing conditions.

Quantification of causal couplings via dynamical effects: a unifying perspective

Quantitative characterization of causal couplings from time series is crucial in studies of complex systems of different origin. Various statistical tools for that exist and new ones are still being developed with a tendency to creating a single, universal, model-free quantifier of coupling strength. However, a clear and generally applicable way of interpreting such universal characteristics is lacking. This work suggests a general conceptual framework for causal coupling quantification, which is based on state space models and extends the concepts of virtual interventions and dynamical causal effects. Namely, two basic kinds of interventions (state space and parametric) and effects (orbital or transient and stationary or limit) are introduced, giving four families of coupling characteristics. The framework provides a unifying view of apparently different well-established measures and allows us to introduce new characteristics, always with a definite "intervention-effect" interpretation. It is shown that diverse characteristics cannot be reduced to any single coupling strength quantifier and their interpretation is inevitably model based. The proposed set of dynamical causal effect measures quantifies different aspects of "how the coupling manifests itself in the dynamics," reformulating the very question about the "causal coupling strength."

Identifying delayed directional couplings with symbolic transfer entropy

We propose a straightforward extension of symbolic transfer entropy to enable the investigation of delayed directional relationships between coupled dynamical systems from time series. Analyzing time series from chaotic model systems, we demonstrate the applicability and limitations of our approach. Our findings obtained from applying our method to infer delayed directed interactions in the human epileptic brain underline the importance of our approach for improving the construction of functional network structures from data.

MuTE: a MATLAB toolbox to compare established and novel estimators of the multivariate transfer entropy

A challenge for physiologists and neuroscientists is to map information transfer between components of the systems that they study at different scales, in order to derive important knowledge on structure and function from the analysis of the recorded dynamics. The components of physiological networks often interact in a nonlinear way and through mechanisms which are in general not completely known. It is then safer that the method of choice for analyzing these interactions does not rely on any model or assumption on the nature of the data and their interactions. Transfer entropy has emerged as a powerful tool to quantify directed dynamical interactions. In this paper we compare different approaches to evaluate transfer entropy, some of them already proposed, some novel, and present their implementation in a freeware MATLAB toolbox. Applications to simulated and real data are presented.

Quantification of the effects of volume conduction on the EEG/MEG connectivity estimates: an index of sensitivity to brain interactions

In the context of EEG/MEG, the term 'volume conduction (VC) effects' refers to the recording of an instantaneous linear mixture of multiple brain source activities by each EEG/MEG channel. VC effects may lead to the detection of spurious functional/effective couplings among EEG/MEG channels that are not caused by brain interactions. It is of importance to determine which detected couplings are indicators of brain interactions and which originate from the VC artefacts. In this paper, a quantitative framework is proposed to explore the origin of detected channel couplings by using two types of surrogate datasets. Also, a sensitivity index (called SI) is proposed to compare the power of different connectivity measures to discriminate between the brain interactions and the instantaneous linear mixing effects. We use seven different functional connectivity estimators to evaluate our method on simulation models and resting state EEG data. The error rate of the proposed framework for simulation data by using each of the connectivity estimators is less than 5.2%. Also, SI ranks these connectivity estimators according to their sensitivity to brain interactions in the presence of VC artefacts. As expected, the connectivity measures which are theoretically robust to VC artefacts yield high SI in simulation models and EEG data. In addition, for EEG data in the alpha frequency band the reproducible functional couplings which are indicators of brain interactions are in the back-front directions. This is consistent with the previous studies in this field.

Sparse model from optimal nonuniform embedding of time series

An approach to obtaining a parsimonious polynomial model from time series is proposed. An optimal minimal nonuniform time series embedding schema is used to obtain a time delay kernel. This scheme recursively optimizes an objective functional that eliminates a maximum number of false nearest neighbors between successive state space reconstruction cycles. A polynomial basis is then constructed from this time delay kernel. A sparse model from this polynomial basis is obtained by solving a regularized least squares problem. The constraint satisfaction problem is made computationally tractable by keeping the ratio between the number of constraints to the number of variables small by using fewer samples spanning all regions of the reconstructed state space. This helps the structure selection process from an exponentially large combinatorial search space. A forward stagewise algorithm is then used for fast discovery of the optimization path. Results are presented for the Mackey-Glass system.

Effect of age on complexity and causality of the cardiovascular control: comparison between model-based and model-free approaches

The proposed approach evaluates complexity of the cardiovascular control and causality among cardiovascular regulatory mechanisms from spontaneous variability of heart period (HP), systolic arterial pressure (SAP) and respiration (RESP). It relies on construction of a multivariate embedding space, optimization of the embedding dimension and a procedure allowing the selection of the components most suitable to form the multivariate embedding space. Moreover, it allows the comparison between linear model-based (MB) and nonlinear model-free (MF) techniques and between MF approaches exploiting local predictability (LP) and conditional entropy (CE). The framework was applied to study age-related modifications of complexity and causality in healthy humans in supine resting (REST) and during standing (STAND). We found that: 1) MF approaches are more efficient than the MB method when nonlinear components are present, while the reverse situation holds in presence of high dimensional embedding spaces; 2) the CE method is the least powerful in detecting age-related trends; 3) the association of HP complexity on age suggests an impairment of cardiac regulation and response to STAND; 4) the relation of SAP complexity on age indicates a gradual increase of sympathetic activity and a reduced responsiveness of vasomotor control to STAND; 5) the association from SAP to HP on age during STAND reveals a progressive inefficiency of baroreflex; 6) the reduced connection from HP to SAP with age might be linked to the progressive exploitation of Frank-Starling mechanism at REST and to the progressive increase of peripheral resistances during STAND; 7) at REST the diminished association from RESP to HP with age suggests a vagal withdrawal and a gradual uncoupling between respiratory activity and heart; 8) the weakened connection from RESP to SAP with age might be related to the progressive increase of left ventricular thickness and vascular stiffness and to the gradual decrease of respiratory sinus arrhythmia.

Direct-coupling information measure from nonuniform embedding

A measure to estimate the direct and directional coupling in multivariate time series is proposed. The measure is an extension of a recently published measure of conditional mutual information from mixed embedding (MIME) for bivariate time series. In the proposed measure of partial MIME (PMIME), the embedding is on all observed variables and it is optimized in explaining the response variable. It is shown that PMIME detects correctly direct coupling and outperforms the (linear) conditional Granger causality and the partial transfer entropy. We demonstrate that PMIME does not rely on significance test and embedding parameters and the number of observed variables has no effect on its statistical accuracy; it may only slow the computations. The importance of these points is shown in simulations and in an application to epileptic multichannel scalp electroencephalograms.

Spurious causalities with transfer entropy

Transfer entropy (TE) seems currently to be the most widely used tool to characterize causal influences in ensembles of complex systems from observed time series. In particular, in an elemental case of two systems, nonzero TEs in both directions are usually interpreted as a sign of a bidirectional coupling. However, one often overlooks that both positive TEs may well be encountered for unidirectionally coupled systems so that a false detection of a causal influence on the basis of a nonzero TE is rather possible. This work highlights typical factors leading to such "spurious couplings": (i) unobserved state variables of the driving system, (ii) low temporal resolution, and (iii) observation errors. All are shown to be particular cases of a general problem: imperfect observations of states of the driving system. Importantly, exact values of TEs, rather than their statistical estimates, are computed here for selected benchmark systems. Conditions for a "spurious" TE to be large and even strongly exceed a "correct" TE are presented and discussed.

Granger causality in cardiovascular variability series: comparison between model-based and model-free approaches

A linear model-based (MB) approach for the evaluation of Granger causality is compared to a nonlinear model-free (MF) one. The MB method is based on the identification of the coefficients of a multivariate linear regression via least-squares procedure. The MF technique is grounded on the concept of local prediction and exploits the k-nearest-neighbors approach. Both the methods optimize the multivariate embedding dimension but MF technique is more parsimonious since the number of components taken from each signal can be different. Both methods were applied to the variability series of heart period (HP), systolic arterial pressure (SAP) and respiration (R) recorded during spontaneous and controlled respiration at 15 breaths/minute (SR and RC15) in 19 healthy humans. Both MB and MF methods revealed the increase of HP predictability during RC15 and the unmodified causality from SAP to HP and from R to HP during RC15, thus suggesting that nonlinear methods are not superior to the linear ones in assessing predictability and causality in healthy humans.

Optimal state-space reconstruction using derivatives on projected manifold

A paradigm for optimal state-space reconstruction with nonuniform time delays is proposed. A comparison based on a diffeomorphic measure and a smoothness cost function shows that the proposed methodology achieves a better reconstruction compared to a reconstruction based on time delays that are multiples of the first minimum of mutual information. It is also shown how the proposed methodology is a more reliable approach to determining the embedding dimension.

Measuring connectivity in linear multivariate processes: definitions, interpretation, and practical analysis

This tutorial paper introduces a common framework for the evaluation of widely used frequency-domain measures of coupling (coherence, partial coherence) and causality (directed coherence, partial directed coherence) from the parametric representation of linear multivariate (MV) processes. After providing a comprehensive time-domain definition of the various forms of connectivity observed in MV processes, we particularize them to MV autoregressive (MVAR) processes and derive the corresponding frequency-domain measures. Then, we discuss the theoretical interpretation of these MVAR-based connectivity measures, showing that each of them reflects a specific time-domain connectivity definition and how this results in the description of peculiar aspects of the information transfer in MV processes. Furthermore, issues related to the practical utilization of these measures on real-time series are pointed out, including MVAR model estimation and significance assessment. Finally, limitations and pitfalls arising from model mis-specification are discussed, indicating possible solutions and providing practical recommendations for a safe computation of the connectivity measures. An example of estimation of the presented measures from multiple EEG signals recorded during a combined visuomotor task is also reported, showing how evaluation of coupling and causality in the frequency domain may help describing specific neurophysiological mechanisms.

Assessing directed interactions from neurophysiological signals--an overview

The study of synchronization phenomena in coupled dynamical systems is an active field of research in many scientific disciplines including the neurosciences. Over the last decades, a number of time series analysis techniques have been proposed to capture both linear and nonlinear aspects of interactions. While most of these techniques allow one to quantify the strength of interactions, developments that resulted from advances in nonlinear dynamics and in information and synchronization theory aim at assessing directed interactions. Most of these techniques, however, assume the underlying systems to be at least approximately stationary and require a large number of data points to robustly assess directed interactions. Recent extensions allow assessing directed interactions from short and transient signals and are particularly suited for the analysis of evoked and event-related activity.

Uniform approach to linear and nonlinear interrelation patterns in multivariate time series

Currently, a variety of linear and nonlinear measures is in use to investigate spatiotemporal interrelation patterns of multivariate time series. Whereas the former are by definition insensitive to nonlinear effects, the latter detect both nonlinear and linear interrelation. In the present contribution we employ a uniform surrogate-based approach, which is capable of disentangling interrelations that significantly exceed random effects and interrelations that significantly exceed linear correlation. The bivariate version of the proposed framework is explored using a simple model allowing for separate tuning of coupling and nonlinearity of interrelation. To demonstrate applicability of the approach to multivariate real-world time series we investigate resting state functional magnetic resonance imaging (rsfMRI) data of two healthy subjects as well as intracranial electroencephalograms (iEEG) of two epilepsy patients with focal onset seizures. The main findings are that for our rsfMRI data interrelations can be described by linear cross-correlation. Rejection of the null hypothesis of linear iEEG interrelation occurs predominantly for epileptogenic tissue as well as during epileptic seizures.

Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique

We present an approach, framed in information theory, to assess nonlinear causality between the subsystems of a whole stochastic or deterministic dynamical system. The approach follows a sequential procedure for nonuniform embedding of multivariate time series, whereby embedding vectors are built progressively on the basis of a minimization criterion applied to the entropy of the present state of the system conditioned to its past states. A corrected conditional entropy estimator compensating for the biasing effect of single points in the quantized hyperspace is used to guarantee the existence of a minimum entropy rate at which to terminate the procedure. The causal coupling is detected according to the Granger notion of predictability improvement, and is quantified in terms of information transfer. We apply the approach to simulations of deterministic and stochastic systems, showing its superiority over standard uniform embedding. Effects of quantization, data length, and noise contamination are investigated. As practical applications, we consider the assessment of cardiovascular regulatory mechanisms from the analysis of heart period, arterial pressure, and respiration time series, and the investigation of the information flow across brain areas from multichannel scalp electroencephalographic recordings.