Permutation entropy of finite-length white-noise time series

A combinatorial view of stochastic processes: White noise

White noise is a fundamental and fairly well understood stochastic process that conforms to the conceptual basis for many other processes, as well as for the modeling of time series. Here, we push a fresh perspective toward white noise that, grounded on combinatorial considerations, contributes to giving new interesting insights both for modeling and theoretical purposes. To this aim, we incorporate the ordinal pattern analysis approach, which allows us to abstract a time series as a sequence of patterns and their associated permutations, and introduce a simple functional over permutations that partitions them into classes encoding their level of asymmetry. We compute the exact probability mass function (p.m.f.) of this functional over the symmetric group of degree n, thus providing the description for the case of an infinite white noise realization. This p.m.f. can be conveniently approximated by a continuous probability density from an exponential family, the Gaussian, hence providing natural sufficient statistics that render a convenient and simple statistical analysis through ordinal patterns. Such analysis is exemplified on experimental data for the spatial increments from tracks of gold nanoparticles in 3D diffusion.

On the Genuine Relevance of the Data-Driven Signal Decomposition-Based Multiscale Permutation Entropy

Ordinal pattern-based approaches have great potential to capture intrinsic structures of dynamical systems, and therefore, they continue to be developed in various research fields. Among these, the permutation entropy (PE), defined as the Shannon entropy of ordinal probabilities, is an attractive time series complexity measure. Several multiscale variants (MPE) have been proposed in order to bring out hidden structures at different time scales. Multiscaling is achieved by combining linear or nonlinear preprocessing with PE calculation. However, the impact of such a preprocessing on the PE values is not fully characterized. In a previous study, we have theoretically decoupled the contribution of specific signal models to the PE values from that induced by the inner correlations of linear preprocessing filters. A variety of linear filters such as the autoregressive moving average (ARMA), Butterworth, and Chebyshev were tested. The current work is an extension to nonlinear preprocessing and especially to data-driven signal decomposition-based MPE. The empirical mode decomposition, variational mode decomposition, singular spectrum analysis-based decomposition and empirical wavelet transform are considered. We identify possible pitfalls in the interpretation of PE values induced by these nonlinear preprocessing, and hence, we contribute to improving the PE interpretation. The simulated dataset of representative processes such as white Gaussian noise, fractional Gaussian processes, ARMA models and synthetic sEMG signals as well as real-life sEMG signals are tested.

Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning

Time series analysis comprises a wide repertoire of methods for extracting information from data sets. Despite great advances in time series analysis, identifying and quantifying the strength of nonlinear temporal correlations remain a challenge. We have recently proposed a new method based on training a machine learning algorithm to predict the temporal correlation parameter, α, of flicker noise (FN) time series. The algorithm is trained using as input features the probabilities of ordinal patterns computed from FN time series, xαFN(t), generated with different values of α. Then, the ordinal probabilities computed from the time series of interest, x(t), are used as input features to the trained algorithm and that returns a value, αe, that contains meaningful information about the temporal correlations present in x(t). We have also shown that the difference, Ω, of the permutation entropy (PE) of the time series of interest, x(t), and the PE of a FN time series generated with α=αe, xαeFN(t), allows the identification of the underlying determinism in x(t). Here, we apply our methodology to different datasets and analyze how αe and Ω correlate with well-known quantifiers of chaos and complexity. We also discuss the limitations for identifying determinism in highly chaotic time series and in periodic time series contaminated by noise. The open source algorithm is available on Github.

Quantum-randomized polarization of laser pulses derived from zero-point diamond motion

Intrinsic randomness in quantum systems is a vital resource for cryptography and other quantum information protocols. To date, randomizing macroscopic polarization states requires randomness from an external source, which is then used to modulate the polarization e.g. for quantum key-distribution protocols. Here, we present a Raman-based device for directly generating laser pulses with quantum-randomized polarizations. We show that crystals of diamond lattice symmetry provide a unique operating point for which the Raman gain is isotropic, so that the spontaneous symmetry breaking initiated by the quantum-random zero-point motion determines the output polarization. Experimentally measured polarizations are demonstrated to be consistent with an independent and identical uniform distribution with an estimated quantum entropy rate of 3.8 bits/pulse.

Entropy and the Brain: An Overview

Entropy is a powerful tool for quantification of the brain function and its information processing capacity. This is evident in its broad domain of applications that range from functional interactivity between the brain regions to quantification of the state of consciousness. A number of previous reviews summarized the use of entropic measures in neuroscience. However, these studies either focused on the overall use of nonlinear analytical methodologies for quantification of the brain activity or their contents pertained to a particular area of neuroscientific research. The present study aims at complementing these previous reviews in two ways. First, by covering the literature that specifically makes use of entropy for studying the brain function. Second, by highlighting the three fields of research in which the use of entropy has yielded highly promising results: the (altered) state of consciousness, the ageing brain, and the quantification of the brain networks' information processing. In so doing, the present overview identifies that the use of entropic measures for the study of consciousness and its (altered) states led the field to substantially advance the previous findings. Moreover, it realizes that the use of these measures for the study of the ageing brain resulted in significant insights on various ways that the process of ageing may affect the dynamics and information processing capacity of the brain. It further reveals that their utilization for analysis of the brain regional interactivity formed a bridge between the previous two research areas, thereby providing further evidence in support of their results. It concludes by highlighting some potential considerations that may help future research to refine the use of entropic measures for the study of brain complexity and its function. The present study helps realize that (despite their seemingly differing lines of inquiry) the study of consciousness, the ageing brain, and the brain networks' information processing are highly interrelated. Specifically, it identifies that the complexity, as quantified by entropy, is a fundamental property of conscious experience, which also plays a vital role in the brain's capacity for adaptation and therefore whose loss by ageing constitutes a basis for diseases and disorders. Interestingly, these two perspectives neatly come together through the association of entropy and the brain capacity for information processing.

Comparative Analysis of the Permutation and Multiscale Entropies for Quantification of the Brain Signal Variability in Naturalistic Scenarios

As alternative entropy estimators, multiscale entropy (MSE) and permutation entropy (PE) are utilized for quantification of the brain function and its signal variability. In this context, their applications are primarily focused on two specific domains: (1) the effect of brain pathology on its function (2) the study of altered states of consciousness. As a result, there is a paucity of research on applicability of these measures in more naturalistic scenarios. In addition, the utility of these measures for quantification of the brain function and with respect to its signal entropy is not well studied. These shortcomings limit the interpretability of the measures when used for quantification of the brain signal entropy. The present study addresses these limitations by comparing MSE and PE with entropy of human subjects' EEG recordings, who watched short movie clips with negative, neutral, and positive content. The contribution of the present study is threefold. First, it identifies a significant anti-correlation between MSE and entropy. In this regard, it also verifies that such an anti-correlation is stronger in the case of negative rather than positive or neutral affects. Second, it finds that MSE significantly differentiates between these three affective states. Third, it observes that the use of PE does not warrant such significant differences. These results highlight the level of association between brain's entropy in response to affective stimuli on the one hand and its quantification in terms of MSE and PE on the other hand. This, in turn, allows for more informed conclusions on the utility of MSE and PE for the study and analysis of the brain signal variability in naturalistic scenarios.

Markov modeling via ordinal partitions: An alternative paradigm for network-based time-series analysis

Mapping time series to complex networks to analyze observables has recently become popular, both at the theoretical and the practitioner's level. The intent is to use network metrics to characterize the dynamics of the underlying system. Applications cover a wide range of problems, from geoscientific measurements to biomedical data and financial time series. It has been observed that different dynamics can produce networks with distinct topological characteristics under a variety of time-series-to-network transforms that have been proposed in the literature. The direct connection, however, remains unclear. Here, we investigate a network transform based on computing statistics of ordinal permutations in short subsequences of the time series, the so-called ordinal partition network. We propose a Markovian framework that allows the interpretation of the network using ergodic-theoretic ideas and demonstrate, via numerical experiments on an ensemble of time series, that this viewpoint renders this technique especially well-suited to nonlinear chaotic signals. The aim is to test the mapping's faithfulness as a representation of the dynamics and the extent to which it retains information from the input data. First, we show that generating networks by counting patterns of increasing length is essentially a mechanism for approximating the analog of the Perron-Frobenius operator in a topologically equivalent higher-dimensional space to the original state space. Then, we illustrate a connection between the connectivity patterns of the networks generated by this mapping and indicators of dynamics such as the hierarchy of unstable periodic orbits embedded within a chaotic attractor. The input is a scalar observable and any projection of a multidimensional flow suffices for reconstruction of the essential dynamics. Additionally, we create a detailed guide for parameter tuning. We argue that there is no optimal value of the pattern length m, rather it admits a scaling region akin to traditional embedding practice. In contrast, the embedding lag and overlap between successive patterns can be chosen exactly in an optimal way. Our analysis illustrates the potential of this transform as a complementary toolkit to traditional time-series methods.

On the Statistical Properties of Multiscale Permutation Entropy: Characterization of the Estimator's Variance

Permutation Entropy (PE) and Multiscale Permutation Entropy (MPE) have been extensively used in the analysis of time series searching for regularities. Although PE has been explored and characterized, there is still a lack of theoretical background regarding MPE. Therefore, we expand the available MPE theory by developing an explicit expression for the estimator’s variance as a function of time scale and ordinal pattern distribution. We derived the MPE Cramér–Rao Lower Bound (CRLB) to test the efficiency of our theoretical result. We also tested our formulation against MPE variance measurements from simulated surrogate signals. We found the MPE variance symmetric around the point of equally probable patterns, showing clear maxima and minima. This implies that the MPE variance is directly linked to the MPE measurement itself, and there is a region where the variance is maximum. This effect arises directly from the pattern distribution, and it is unrelated to the time scale or the signal length. The MPE variance also increases linearly with time scale, except when the MPE measurement is close to its maximum, where the variance presents quadratic growth. The expression approaches the CRLB asymptotically, with fast convergence. The theoretical variance is close to the results from simulations, and appears consistently below the actual measurements. By knowing the MPE variance, it is possible to have a clear precision criterion for statistical comparison in real-life applications.

A Hybrid De-Noising Algorithm for the Gear Transmission System Based on CEEMDAN-PE-TFPF

In order to remove noise and preserve the important features of a signal, a hybrid de-noising algorithm based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Permutation Entropy (PE), and Time-Frequency Peak Filtering (TFPF) is proposed. In view of the limitations of the conventional TFPF method regarding the fixed window length problem, CEEMDAN and PE are applied to compensate for this, so that the signal is balanced with respect to both noise suppression and signal fidelity. First, the Intrinsic Mode Functions (IMFs) of the original spectra are obtained using the CEEMDAN algorithm, and the PE value of each IMF is calculated to classify whether the IMF requires filtering, then, for different IMFs, we select different window lengths to filter them using TFPF; finally, the signal is reconstructed as the sum of the filtered and residual IMFs. The filtering results of a simulated and an actual gearbox vibration signal verify that the de-noising results of CEEMDAN-PE-TFPF outperforms other signal de-noising methods, and the proposed method can reveal fault characteristic information effectively.

Permutation entropy with vector embedding delays

Permutation entropy (PE) is a statistic used widely for the detection of structure within a time series. Embedding delay times at which the PE is reduced are characteristic timescales for which such structure exists. Here, a generalized scheme is investigated where embedding delays are represented by vectors rather than scalars, permitting PE to be calculated over a (D-1)-dimensional space, where D is the embedding dimension. This scheme is applied to numerically generated noise, sine wave and logistic map series, and experimental data sets taken from a vertical-cavity surface emitting laser exhibiting temporally localized pulse structures within the round-trip time of the laser cavity. Results are visualized as PE maps as a function of embedding delay, with low PE values indicating combinations of embedding delays where correlation structure is present. It is demonstrated that vector embedding delays enable identification of structure that is ambiguous or masked, when the embedding delay is constrained to scalar form.

Variance of permutation entropy and the influence of ordinal pattern selection

Permutation entropy (PE) is a widely used measure for complexity, often used to distinguish between complex systems (or complex systems in different states). Here, the PE variance for a stationary time series is derived, and the influence of ordinal pattern selection, specifically whether the ordinal patterns are permitted to overlap or not, is examined. It was found that permitting ordinal patterns to overlap reduces the PE variance, improving the ability of this statistic to distinguish between complex system states for both numeric (fractional Gaussian noise) and experimental (semiconductor laser with optical feedback) systems. However, with overlapping ordinal patterns, the precision to which the PE variance can be estimated becomes diminished, which can manifest as increased incidences of false positive and false negative errors when applying PE to statistical inference problems.