The mutual information: detecting and evaluating dependencies between variables

MOTIVATION

Clustering co-expressed genes usually requires the definition of 'distance' or 'similarity' between measured datasets, the most common choices being Pearson correlation or Euclidean distance. With the size of available datasets steadily increasing, it has become feasible to consider other, more general, definitions as well. One alternative, based on information theory, is the mutual information, providing a general measure of dependencies between variables. While the use of mutual information in cluster analysis and visualization of large-scale gene expression data has been suggested previously, the earlier studies did not focus on comparing different algorithms to estimate the mutual information from finite data.

RESULTS

Here we describe and review several approaches to estimate the mutual information from finite datasets. Our findings show that the algorithms used so far may be quite substantially improved upon. In particular when dealing with small datasets, finite sample effects and other sources of potentially misleading results have to be taken into account.

Quantitative analysis of heart rate variability

In the modern industrialized countries every year several hundred thousands of people die due to sudden cardiac death. The individual risk for this sudden cardiac death cannot be defined precisely by common available, noninvasive diagnostic tools like Holter monitoring, highly amplified ECG and traditional linear analysis of heart rate variability (HRV). Therefore, we apply some rather unconventional methods of nonlinear dynamics to analyze the HRV. Especially, some complexity measures that are based on symbolic dynamics as well as a new measure, the renormalized entropy, detect some abnormalities in the HRV of several patients who have been classified in the low risk group by traditional methods. A combination of these complexity measures with the parameters in the frequency domain seems to be a promising way to get a more precise definition of the individual risk. These findings have to be validated by a representative number of patients. (c) 1995 American Institute of Physics.

Observing and interpreting correlations in metabolomic networks

MOTIVATION

Metabolite profiling aims at an unbiased identification and quantification of all the metabolites present in a biological sample. Based on their pair-wise correlations, the data obtained from metabolomic experiments are organized into metabolic correlation networks and the key challenge is to deduce unknown pathways based on the observed correlations. However, the data generated is fundamentally different from traditional biological measurements and thus the analysis is often restricted to rather pragmatic approaches, such as data mining tools, to discriminate between different metabolic phenotypes.

METHODS AND RESULTS

We investigate to what extent the data generated networks reflect the structure of the underlying biochemical pathways. The purpose of this work is 2-fold: Based on the theory of stochastic systems, we first introduce a framework which shows that the emergent correlations can be interpreted as a 'fingerprint' of the underlying biophysical system. This result leads to a systematic relationship between observed correlation networks and the underlying biochemical pathways. In a second step, we investigate to what extent our result is applicable to the problem of reverse engineering, i.e. to recover the underlying enzymatic reaction network from data. The implications of our findings for other bioinformatics approaches are discussed.

Synchronization in the human cardiorespiratory system

We investigate synchronization between cardiovascular and respiratory systems in healthy humans under free-running conditions. For this aim we analyze nonstationary irregular bivariate data, namely, electrocardiograms and measurements of respiratory flow. We briefly discuss a statistical approach to synchronization in noisy and chaotic systems and illustrate it with numerical examples; effects of phase and frequency locking are considered. Next, we present and discuss methods suitable for the detection of hidden synchronous epochs from such data. The analysis of the experimental records reveals synchronous regimes of different orders n:m and transitions between them; the physiological significance of this finding is discussed.

Array-enhanced coherence resonance: nontrivial effects of heterogeneity and spatial independence of noise

We demonstrate the effect of coherence resonance in a heterogeneous array of coupled Fitz Hugh-Nagumo neurons. It is shown that coupling of such elements leads to a significantly stronger coherence compared to that of a single element. We report nontrivial effects of parameter heterogeneity and spatial independence of noise on array-enhanced coherence resonance; especially, we find that (i) the coherence increases as spatial correlation of the noise decreases, and (ii) inhomogeneity in the parameters of the array enhances the coherence. Our results have the implication that generic heterogeneity and background noise can play a constructive role to enhance the time precision of firing in neural systems.

Short-term forecasting of life-threatening cardiac arrhythmias based on symbolic dynamics and finite-time growth rates

Ventricular tachycardia or fibrillation (VT-VF) as fatal cardiac arrhythmias are the main factors triggering sudden cardiac death. The objective of this study is to find early signs of sustained VT-VF in patients with an implanted cardioverter-defibrillator (ICD). These devices are able to safeguard patients by returning their hearts to a normal rhythm via strong defibrillatory shocks; additionally, they store the 1000 beat-to-beat intervals immediately before the onset of a life-threatening arrhythmia. We study these 1000 beat-to-beat intervals of 17 chronic heart failure ICD patients before the onset of a life-threatening arrhythmia and at a control time, i.e., without a VT-VF event. To characterize these rather short data sets, we calculate heart rate variability parameters from the time and frequency domain, from symbolic dynamics as well as the finite-time growth rates. We find that neither the time nor the frequency domain parameters show significant differences between the VT-VF and the control time series. However, two parameters from symbolic dynamics as well as the finite-time growth rates discriminate significantly both groups. These findings could be of importance in algorithms for next generation ICD's to improve the diagnostics and therapy of VT-VF.

Multiparametric analysis of heart rate variability used for risk stratification among survivors of acute myocardial infarction

A multiparametric heart rate variability analysis was performed to prove if combined heart rate variability (HRV) measures of different domains improve the result of risk stratification in patients after myocardial infarction. In this study, standard time domain, frequency domain and non-linear dynamics measures of HRV assessment were applied to 572 survivors of acute myocardial infarction. Three parameter sets each consisting of 4 parameters were applied and compared with the standard measurement of global heart rate variability HRVi. Discriminant analysis technique and t-test were performed to separate the high risk groups from the survivors. The predictive value of this approach was evaluated with receiver operator (ROC) and positive predictive accuracy (PPA) curves. Results--The discriminant analysis shows a separation of patients suffered by all cause mortality in 80% (best single parameter 74%) and sudden arrhythmic death in 86% (73%). All parameters of set 1 show a high significant difference (p < 0.001) between survivors and non-survivors based on two-tailed t-test. The specificity level of the multivariate parameter sets is at the 70% sensitivity level (ROC) about 85-90%, whereas HRVi shows maximum levels of 70%. The PPA in the all cause mortality group is at the 70% sensitivity level twice as high as the univariate HRV measure and increases to more than fourfold as high within the VT/VF group. In conclusion, in this population, the multiparametric approach with the combination of four parameters from all domains especially from NLD seems to be a better predictor of high arrhythmia risk than the standard measurement of global heart rate variability.

Phase synchronization in the forced Lorenz system

We demonstrate that the dynamics of phase synchronization in a chaotic system under weak periodic forcing depends crucially on the distribution of intrinsic characteristic times of this system. Under the external periodic action, the frequency of every unstable periodic orbit is locked to the frequency of the force. In systems which in the autonomous case displays nearly isochronous chaotic rotations, the locking ratio is the same for all periodic orbits; since a typical chaotic orbit wanders between the periodic ones, its phase follows the phase of the force. For the Lorenz attractor with its unbounded times of return onto a Poincaré surface, such state of perfect phase synchronization is inaccessible. Analysis with the help of unstable periodic orbits shows that this state is replaced by another one, which we call "imperfect phase synchronization," and in which we observe alternation of temporal segments, corresponding to different rational values of frequency lockings.

Explosive synchronization in a general complex network

Explosive synchronization (ES) has recently attracted much attention, where its two necessary conditions are found to be a scale-free network topology and a positive correlation between the natural frequencies of the oscillators and their degrees. Here we present a framework for ES to be observed in a general complex network, where a positive correlation between coupling strengths of the oscillators and the absolute of their natural frequencies is assumed and the previous studies are included as specific cases. In the framework, the previous two necessary conditions are replaced by another one, thus fundamentally deepening the understanding of the microscopic mechanism toward synchronization. A rigorous analytical treatment by a mean field is provided to explain the mechanism of ES in this alternate framework.

Interpreting correlations in metabolomic networks

Correlations, as observed between the concentrations of metabolites in a biological sample, may be used to gain additional information about the physiological state of a given tissue. In this mini-review, we discuss the integration of these observed correlations into metabolomic networks and their relationships with the underlying biochemical pathways.

Experimental evidence, numerics, and theory of vibrational resonance in bistable systems

We consider an overdamped bistable oscillator subject to the action of a biharmonic force with very different frequencies, and study the response of the system when the parameters of the high-frequency force are varied. A resonantlike behavior is obtained when the amplitude or the frequency of this force is modified in an experiment performed by means of an analog circuit. This behavior, confirmed by numerical simulations, is explained on the basis of a theoretical approach.

Nonstationary Gaussian processes in wavelet domain: synthesis, estimation, and significance testing

We propose an equivalence class of nonstationary Gaussian stochastic processes defined in the wavelet domain. These processes are characterized by means of wavelet multipliers and exhibit well-defined time-dependent spectral properties. They allow one to generate realizations of any wavelet spectrum. Based on this framework, we study the estimation of continuous wavelet spectra, i.e., we calculate variance and bias of arbitrary estimated continuous wavelet spectra. Finally, we develop an areawise significance test for continuous wavelet spectra to overcome the difficulties of multiple testing; it uses basic properties of continuous wavelet transform to decide whether a pointwise significant result is a real feature of the process or indistinguishable from typical stochastic fluctuations. This test is compared to the conventional one in terms of sensitivity and specificity. A software package for continuous wavelet spectral analysis and synthesis is presented.

Remote synchronization in star networks

We study phase synchronization in a network motif with a starlike structure in which the central node's (the hub's) frequency is strongly detuned against the other peripheral nodes. We find numerically and experimentally a regime of remote synchronization (RS), where the peripheral nodes form a phase synchronized cluster, while the hub remains free with its own dynamics and serves just as a transmitter for the other nodes. We explain the mechanism for this RS by the existence of a free amplitude and also show that systems with a fixed or constant amplitude, such as the classic Kuramoto phase oscillator, are not able to generate this phenomenon. Further, we derive an analytic expression which supports our explanation of the mechanism.

Inner composition alignment for inferring directed networks from short time series

Identifying causal links (couplings) is a fundamental problem that facilitates the understanding of emerging structures in complex networks. We propose and analyze inner composition alignment-a novel, permutation-based asymmetric association measure to detect regulatory links from very short time series, currently applied to gene expression. The measure can be used to infer the direction of couplings, detect indirect (superfluous) links, and account for autoregulation. Applications to the gene regulatory network of E. coli are presented.

Improved analysis of heart rate variability by methods of nonlinear dynamics

The traditional analysis of heart rate variability (HRV) in the time and frequency domains seems to be an independent predictive marker for sudden cardiac death. Because the usual applied methods of HRV analysis describe only linear or strong periodic phenomena, the authors have developed new methods of HRV analysis based on nonlinear dynamics. In that way, parameters are extracted that quantify more complex processes and their complicated relationships. These methods are symbolic dynamics that describes the beat-to-beat dynamics and renormalized entropy that compares the complexity of power spectra on a normalized energy level. In an initial investigation, the HRV of 35 healthy subjects and 39 cardiac patients have been analyzed. Using discriminant functions, the authors found an optimal (100%) differentiation between the group of healthy subjects (even using only an age-matched subgroup of 12 subjects) and that of patients after myocardial infarction with a high electrical risk (Lown 4b). Applying this discriminant function to a group of patients with low electrical risk, four patients show the same behavior indicative of a high risk score, which might be a sign for a hidden high risk, two patients show healthy behavior, and the remaining patients show a separate pattern. The use of new methods of nonlinear dynamics in combination with parameters of the time and frequency domains in HRV offers possibilities for improved classification of HRV behavior. It is suggested that this could lead to a more detailed classification of individual high risk.