ordpy: A Python package for data analysis with permutation entropy and ordinal network methods

Rényi entropy-complexity causality space: a novel neurocomputational tool for detecting scale-free features in EEG/iEEG data

Scale-free brain activity, linked with learning, the integration of different time scales, and the formation of mental models, is correlated with a metastable cognitive basis. The spectral slope, a key aspect of scale-free dynamics, was proposed as a potential indicator to distinguish between different sleep stages. Studies suggest that brain networks maintain a consistent scale-free structure across wakefulness, anesthesia, and recovery. Although differences in anesthetic sensitivity between the sexes are recognized, these variations are not evident in clinical electroencephalographic recordings of the cortex. Recently, changes in the slope of the power law exponent of neural activity were found to correlate with changes in Rényi entropy, an extended concept of Shannon's information entropy. These findings establish quantifiers as a promising tool for the study of scale-free dynamics in the brain. Our study presents a novel visual representation called the Rényi entropy-complexity causality space, which encapsulates complexity, permutation entropy, and the Rényi parameter q. The main goal of this study is to define this space for classical dynamical systems within theoretical bounds. In addition, the study aims to investigate how well different time series mimicking scale-free activity can be discriminated. Finally, this tool is used to detect dynamic features in intracranial electroencephalography (iEEG) signals. To achieve these goals, the study implementse the Bandt and Pompe method for ordinal patterns. In this process, each signal is associated with a probability distribution, and the causal measures of Rényi entropy and complexity are computed based on the parameter q. This method is a valuable tool for analyzing simulated time series. It effectively distinguishes elements of correlated noise and provides a straightforward means of examining differences in behaviors, characteristics, and classifications. For the iEEG experimental data, the REM state showed a greater number of significant sex-based differences, while the supramarginal gyrus region showed the most variation across different modes and analyzes. Exploring scale-free brain activity with this framework could provide valuable insights into cognition and neurological disorders. The results may have implications for understanding differences in brain function between the sexes and their possible relevance to neurological disorders.

Triadic percolation induces dynamical topological patterns in higher-order networks

Triadic interactions are higher-order interactions which occur when a set of nodes affects the interaction between two other nodes. Examples of triadic interactions are present in the brain when glia modulate the synaptic signals among neuron pairs or when interneuron axo-axonic synapses enable presynaptic inhibition and facilitation, and in ecosystems when one or more species can affect the interaction among two other species. On random graphs, triadic percolation has been recently shown to turn percolation into a fully fledged dynamical process in which the size of the giant component undergoes a route to chaos. However, in many real cases, triadic interactions are local and occur on spatially embedded networks. Here, we show that triadic interactions in spatial networks induce a very complex spatio-temporal modulation of the giant component which gives rise to triadic percolation patterns with significantly different topology. We classify the observed patterns (stripes, octopus, and small clusters) with topological data analysis and we assess their information content (entropy and complexity). Moreover, we illustrate the multistability of the dynamics of the triadic percolation patterns, and we provide a comprehensive phase diagram of the model. These results open new perspectives in percolation as they demonstrate that in presence of spatial triadic interactions, the giant component can acquire a time-varying topology. Hence, this work provides a theoretical framework that can be applied to model realistic scenarios in which the giant component is time dependent as in neuroscience.

Characterizing Complex Spatiotemporal Patterns from Entropy Measures

In addition to their importance in statistical thermodynamics, probabilistic entropy measurements are crucial for understanding and analyzing complex systems, with diverse applications in time series and one-dimensional profiles. However, extending these methods to two- and three-dimensional data still requires further development. In this study, we present a new method for classifying spatiotemporal processes based on entropy measurements. To test and validate the method, we selected five classes of similar processes related to the evolution of random patterns: (i) white noise; (ii) red noise; (iii) weak turbulence from reaction to diffusion; (iv) hydrodynamic fully developed turbulence; and (v) plasma turbulence from MHD. Considering seven possible ways to measure entropy from a matrix, we present the method as a parameter space composed of the two best separating measures of the five selected classes. The results highlight better combined performance of Shannon permutation entropy (SHp) and a new approach based on Tsallis Spectral Permutation Entropy (Sqs). Notably, our observations reveal the segregation of reaction terms in this SHp×Sqs space, a result that identifies specific sectors for each class of dynamic process, and it can be used to train machine learning models for the automatic classification of complex spatiotemporal patterns.

Multifractal analysis of maize and soybean DNA

This paper investigates the complexity of DNA sequences in maize and soybean using the multifractal detrended fluctuation analysis (MF-DFA) method, chaos game representation (CGR), and the complexity-entropy plane approach. The study aims to understand the patterns and structures of these DNA sequences, which can provide insights into their genetic makeup and improve crop yield and quality. The results show that maize and soybean DNA sequences exhibit fractal properties, indicating a complex and self-organizing structure. We observe the persistence trend between sequences of base pairs, which indicates long-range correlations between base pairs. We also identified the stochastic nature of the DNA sequences of both species.

Characterizing unstructured data with the nearest neighbor permutation entropy

Permutation entropy and its associated frameworks are remarkable examples of physics-inspired techniques adept at processing complex and extensive datasets. Despite substantial progress in developing and applying these tools, their use has been predominantly limited to structured datasets such as time series or images. Here, we introduce the k-nearest neighbor permutation entropy, an innovative extension of the permutation entropy tailored for unstructured data, irrespective of their spatial or temporal configuration and dimensionality. Our approach builds upon nearest neighbor graphs to establish neighborhood relations and uses random walks to extract ordinal patterns and their distribution, thereby defining the k-nearest neighbor permutation entropy. This tool not only adeptly identifies variations in patterns of unstructured data but also does so with a precision that significantly surpasses conventional measures such as spatial autocorrelation. Additionally, it provides a natural approach for incorporating amplitude information and time gaps when analyzing time series or images, thus significantly enhancing its noise resilience and predictive capabilities compared to the usual permutation entropy. Our research substantially expands the applicability of ordinal methods to more general data types, opening promising research avenues for extending the permutation entropy toolkit for unstructured data.

Bayesian at heart: Towards autonomic outflow estimation via generative state-space modelling of heart rate dynamics

Recent research is revealing how cognitive processes are supported by a complex interplay between the brain and the rest of the body, which can be investigated by the analysis of physiological features such as breathing rhythms, heart rate, and skin conductance. Heart rate dynamics are of particular interest as they provide a way to track the sympathetic and parasympathetic outflow from the autonomic nervous system, which is known to play a key role in modulating attention, memory, decision-making, and emotional processing. However, extracting useful information from heartbeats about the autonomic outflow is still challenging due to the noisy estimates that result from standard signal-processing methods. To advance this state of affairs, we propose a novel approach in how to conceptualise and model heart rate: instead of being a mere summary of the observed inter-beat intervals, we introduce a modelling framework that views heart rate as a hidden stochastic process that drives the observed heartbeats. Moreover, by leveraging the rich literature of state-space modelling and Bayesian inference, our proposed framework delivers a description of heart rate dynamics that is not a point estimate but a posterior distribution of a generative model. We illustrate the capabilities of our method by showing that it recapitulates linear properties of conventional heart rate estimators, while exhibiting a better discriminative power for metrics of dynamical complexity compared across different physiological states.

Daily Streamflow of Argentine Rivers Analysis Using Information Theory Quantifiers

This paper analyzes the temporal evolution of streamflow for different rivers in Argentina based on information quantifiers such as statistical complexity and permutation entropy. The main objective is to identify key details of the dynamics of the analyzed time series to differentiate the degrees of randomness and chaos. The permutation entropy is used with the probability distribution of ordinal patterns and the Jensen-Shannon divergence to calculate the disequilibrium and the statistical complexity. Daily streamflow series at different river stations were analyzed to classify the different hydrological systems. The complexity-entropy causality plane (CECP) and the representation of the Shannon entropy and Fisher information measure (FIM) show that the daily discharge series could be approximately represented with Gaussian noise, but the variances highlight the difficulty of modeling a series of natural phenomena. An analysis of stations downstream from the Yacyretá dam shows that the operation affects the randomness of the daily discharge series at hydrometric stations near the dam. When the station is further downstream, however, this effect is attenuated. Furthermore, the size of the basin plays a relevant role in modulating the process. Large catchments have smaller values for entropy, and the signal is less noisy due to integration over larger time scales. In contrast, small and mountainous basins present a rapid response that influences the behavior of daily discharge while presenting a higher entropy and lower complexity. The results obtained in the present study characterize the behavior of the daily discharge series in Argentine rivers and provide key information for hydrological modeling.

Hard c-mean transition network method for analysis of time series

Transition network is a powerful tool to analyze nonlinear dynamic characteristics of complex systems, which characterizes the temporal transition property. Few, if any, existing approaches map different time series into transition networks with the same size so that temporal information of time series can be captured more effectively by network measures including typical average node degree, average path length, and so on. To construct a fixed size transition network, the proposed approach uses the embedding dimension method to reconstruct phase space from time series and divides state vectors into different nodes based on the hard c-mean clustering algorithm. The links are determined by the temporal succession of nodes. Our novel method is illustrated by three case studies: distinction of different dynamic behaviors, detection of parameter perturbation of dynamical system, and identification of seismic airgun based on sound data recorded in central Atlantic Ocean. The results show that our proposed method shows good performance in capturing the underlying nonlinear and nonstationary dynamics from short and noisy time series.

Chaotic dynamics of the Hénon map and neuronal input-output: A comparison with neurophysiological data

In this study, the Hénon map was analyzed using quantifiers from information theory in order to compare its dynamics to experimental data from brain regions known to exhibit chaotic behavior. The goal was to investigate the potential of the Hénon map as a model for replicating chaotic brain dynamics in the treatment of Parkinson's and epilepsy patients. The dynamic properties of the Hénon map were compared with data from the subthalamic nucleus, the medial frontal cortex, and a q-DG model of neuronal input-output with easy numerical implementation to simulate the local behavior of a population. Using information theory tools, Shannon entropy, statistical complexity, and Fisher's information were analyzed, taking into account the causality of the time series. For this purpose, different windows over the time series were considered. The findings revealed that neither the Hénon map nor the q-DG model could perfectly replicate the dynamics of the brain regions studied. However, with careful consideration of the parameters, scales, and sampling used, they were able to model some characteristics of neural activity. According to these results, normal neural dynamics in the subthalamic nucleus region may present a more complex spectrum within the complexity-entropy causality plane that cannot be represented by chaotic models alone. The dynamic behavior observed in these systems using these tools is highly dependent on the studied temporal scale. As the size of the sample studied increases, the dynamics of the Hénon map become increasingly different from those of biological and artificial neural systems.

Multiscale two-dimensional permutation entropy to analyze encrypted images

Multiscale versions of weighted (and non-weighted) permutation entropy for two dimensions are considered in order to compare and analyze the results when different experiments are conducted. We propose the application of these measures to analyze encrypted images with different security levels and encryption methods.

Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View

An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the "black-box" nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity-entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework.

The effect of age on electroencephalogram measures of anesthesia hypnosis: A comparison of BIS, Alpha Power, Lempel-Ziv complexity and permutation entropy during propofol induction

Background

Improving anesthesia administration for elderly population is of particular importance because they undergo considerably more surgical procedures and are at the most risk of suffering from anesthesia-related complications. Intraoperative brain monitors electroencephalogram (EEG) have proved useful in the general population, however, in elderly subjects this is contentious. Probably because these monitors do not account for the natural differences in EEG signals between young and older patients. In this study we attempted to systematically characterize the age-dependence of different EEG measures of anesthesia hypnosis.

Methods

We recorded EEG from 30 patients with a wide age range (19–99 years old) and analyzed four different proposed indexes of depth of hypnosis before, during and after loss of behavioral response due to slow propofol infusion during anesthetic induction. We analyzed Bispectral Index (BIS), Alpha Power and two entropy-related EEG measures, Lempel-Ziv complexity (LZc), and permutation entropy (PE) using mixed-effect analysis of variances (ANOVAs). We evaluated their possible age biases and their trajectories during propofol induction.

Results

All measures were dependent on anesthesia stages. BIS, LZc, and PE presented lower values at increasing anesthetic dosage. Inversely, Alpha Power increased with increasing propofol at low doses, however this relation was reversed at greater effect-site propofol concentrations. Significant group differences between elderly patients (>65 years) and young patients were observed for BIS, Alpha Power, and LZc, but not for PE.

Conclusion

BIS, Alpha Power, and LZc show important age-related biases during slow propofol induction. These should be considered when interpreting and designing EEG monitors for clinical settings. Interestingly, PE did not present significant age differences, which makes it a promising candidate as an age-independent measure of hypnotic depth to be used in future monitor development.

Estimating Permutation Entropy Variability via Surrogate Time Series

In the last decade permutation entropy (PE) has become a popular tool to analyze the degree of randomness within a time series. In typical applications, changes in the dynamics of a source are inferred by observing changes of PE computed on different time series generated by that source. However, most works neglect the crucial question related to the statistical significance of these changes. The main reason probably lies in the difficulty of assessing, out of a single time series, not only the PE value, but also its uncertainty. In this paper we propose a method to overcome this issue by using generation of surrogate time series. The analysis conducted on both synthetic and experimental time series shows the reliability of the approach, which can be promptly implemented by means of widely available numerical tools. The method is computationally affordable for a broad range of users.

Permutation Jensen-Shannon distance: A versatile and fast symbolic tool for complex time-series analysis

The main motivation of this paper is to introduce the permutation Jensen-Shannon distance, a symbolic tool able to quantify the degree of similarity between two arbitrary time series. This quantifier results from the fusion of two concepts, the Jensen-Shannon divergence and the encoding scheme based on the sequential ordering of the elements in the data series. The versatility and robustness of this ordinal symbolic distance for characterizing and discriminating different dynamics are illustrated through several numerical and experimental applications. Results obtained allow us to be optimistic about its usefulness in the field of complex time-series analysis. Moreover, thanks to its simplicity, low computational cost, wide applicability, and less susceptibility to outliers and artifacts, this ordinal measure can efficiently handle large amounts of data and help to tackle the current big data challenges.