Machine-Learning Optimized Measurements of Chaotic Dynamical Systems via the Information Bottleneck

Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is challenging and has generally required intimate knowledge of the dynamics in the few cases where it has been done. We establish an equivalence between a perfect measurement and a variant of the information bottleneck. As a consequence, we can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data. We obtain approximately optimal measurements for multiple chaotic maps and lay the necessary groundwork for efficient information extraction from general time series.

Bayesian estimation of the Kullback-Leibler divergence for categorical systems using mixtures of Dirichlet priors

In many applications in biology, engineering, and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical divergences quantify the difference between two distributions. However, their estimation is very difficult and empirical methods often fail, especially when the samples are small. We develop a Bayesian estimator of the Kullback-Leibler divergence between two probability distributions that makes use of a mixture of Dirichlet priors on the distributions being compared. We study the properties of the estimator on two examples: probabilities drawn from Dirichlet distributions and random strings of letters drawn from Markov chains. We extend the approach to the squared Hellinger divergence. Both estimators outperform other estimation techniques, with better results for data with a large number of categories and for higher values of divergences.

Selecting an Effective Entropy Estimator for Short Sequences of Bits and Bytes with Maximum Entropy

Entropy makes it possible to measure the uncertainty about an information source from the distribution of its output symbols. It is known that the maximum Shannon's entropy of a discrete source of information is reached when its symbols follow a Uniform distribution. In cryptography, these sources have great applications since they allow for the highest security standards to be reached. In this work, the most effective estimator is selected to estimate entropy in short samples of bytes and bits with maximum entropy. For this, 18 estimators were compared. Results concerning the comparisons published in the literature between these estimators are discussed. The most suitable estimator is determined experimentally, based on its bias, the mean square error short samples of bytes and bits.

Symbolic partition in chaotic maps

In this work, we only use data on the unstable manifold to locate the partition boundaries by checking folding points at different levels, which practically coincide with homoclinic tangencies. The method is then applied to the classic two-dimensional Hénon map and a well-known three-dimensional map. Comparison with previous results is made in the Hénon case, and Lyapunov exponents are computed through the metric entropy based on the partition to show the validity of the current scheme.

Estimating entropy rate from censored symbolic time series: A test for time-irreversibility

In this work, we introduce a method for estimating the entropy rate and the entropy production rate from a finite symbolic time series. From the point of view of statistics, estimating entropy from a finite series can be interpreted as a problem of estimating parameters of a distribution with a censored or truncated sample. We use this point of view to give estimations of the entropy rate and the entropy production rate, assuming that they are parameters of a (limit) distribution. The last statement is actually a consequence of the fact that the distribution of estimations obtained from recurrence-time statistics satisfies the central limit theorem. We test our method using a time series coming from Markov chain models, discrete-time chaotic maps, and a real DNA sequence from the human genome.