Statistical properties of the entropy from ordinal patterns

The asymptotic distribution of the permutation entropy

Ordinal patterns serve as a robust symbolic transformation technique, enabling the unveiling of latent dynamics within time series data. This methodology involves constructing histograms of patterns, followed by the calculation of both entropy and statistical complexity-an avenue yet to be fully understood in terms of its statistical properties. While asymptotic results can be derived by assuming a multinomial distribution for histogram proportions, the challenge emerges from the non-independence present in the sequence of ordinal patterns. Consequently, the direct application of the multinomial assumption is questionable. This study focuses on the computation of the asymptotic distribution of permutation entropy, considering the inherent patterns' correlation structure. Furthermore, the research delves into a comparative analysis, pitting this distribution against the entropy derived from a multinomial law. We present simulation algorithms for sampling time series with prescribed histograms of patterns and transition probabilities between them. Through this analysis, we better understand the intricacies of ordinal patterns and their statistical attributes.

Asymptotic Distribution of Certain Types of Entropy under the Multinomial Law

We obtain expressions for the asymptotic distributions of the Rényi and Tsallis of order q entropies and Fisher information when computed on the maximum likelihood estimator of probabilities from multinomial random samples. We verify that these asymptotic models, two of which (Tsallis and Fisher) are normal, describe well a variety of simulated data. In addition, we obtain test statistics for comparing (possibly different types of) entropies from two samples without requiring the same number of categories. Finally, we apply these tests to social survey data and verify that the results are consistent but more general than those obtained with a χ2 test.