Distinguishing chaotic and stochastic dynamics from time series by using a multiscale symbolic approach

Fetal QRS extraction from single-channel abdominal ECG using adaptive improved permutation entropy

Fetal electrocardiogram (fECG) monitoring is crucial for assessing fetal condition during pregnancy. However, current fECG extraction algorithms are not suitable for wearable devices due to their high computational cost and multi-channel signal requirement. The paper introduces a novel and efficient algorithm called Adaptive Improved Permutation Entropy (AIPE), which can extract fetal QRS from a single-channel abdominal ECG (aECG). The proposed algorithm is robust and computationally efficient, making it a reliable and effective solution for wearable devices. To evaluate the performance of the proposed algorithm, we utilized our clinical data obtained from a pilot study with 10 subjects, each recording lasting 20 min. Additionally, data from the PhysioNet 2013 Challenge bank with labeled QRS complex annotations were simulated. The proposed methodology demonstrates an average positive predictive value ( + P ) of 91.0227%, sensitivity (Se) of 90.4726%, and F1 score of 90.6525% from the PhysioNet 2013 Challenge bank, outperforming other methods. The results suggest that AIPE could enable continuous home-based monitoring of unborn babies, even when mothers are not engaging in any hard physical activities.

Revisiting the Characterization of Resting Brain Dynamics with the Permutation Jensen-Shannon Distance

Taking into account the complexity of the human brain dynamics, the appropriate characterization of any brain state is a challenge not easily met. Actually, even the discrimination of simple behavioral tasks, such as resting with eyes closed or eyes open, represents an intricate problem and many efforts have been and are being made to overcome it. In this work, the aforementioned issue is carefully addressed by performing multiscale analyses of electroencephalogram records with the permutation Jensen-Shannon distance. The influence that linear and nonlinear temporal correlations have on the discrimination is unveiled. Results obtained lead to significant conclusions that help to achieve an improved distinction between these resting brain states.

Characterizing unstructured data with the nearest neighbor permutation entropy

Permutation entropy and its associated frameworks are remarkable examples of physics-inspired techniques adept at processing complex and extensive datasets. Despite substantial progress in developing and applying these tools, their use has been predominantly limited to structured datasets such as time series or images. Here, we introduce the k-nearest neighbor permutation entropy, an innovative extension of the permutation entropy tailored for unstructured data, irrespective of their spatial or temporal configuration and dimensionality. Our approach builds upon nearest neighbor graphs to establish neighborhood relations and uses random walks to extract ordinal patterns and their distribution, thereby defining the k-nearest neighbor permutation entropy. This tool not only adeptly identifies variations in patterns of unstructured data but also does so with a precision that significantly surpasses conventional measures such as spatial autocorrelation. Additionally, it provides a natural approach for incorporating amplitude information and time gaps when analyzing time series or images, thus significantly enhancing its noise resilience and predictive capabilities compared to the usual permutation entropy. Our research substantially expands the applicability of ordinal methods to more general data types, opening promising research avenues for extending the permutation entropy toolkit for unstructured data.

Quantifying the diversity of multiple time series with an ordinal symbolic approach

The main motivation of this paper is to introduce the ordinal diversity, a symbolic tool able to quantify the degree of diversity of multiple time series. Analytical, numerical, and experimental analyses illustrate the utility of this measure to quantify how diverse, from an ordinal perspective, a set of many time series is. We have shown that ordinal diversity is able to characterize dynamical richness and dynamical transitions in stochastic processes and deterministic systems, including chaotic regimes. This ordinal tool also serves to identify optimal operating conditions in the machine learning approach of reservoir computing. These results allow us to envision potential applications for the handling and characterization of large amounts of data, paving the way for addressing some of the most pressing issues facing the current big data paradigm.

Ordinal pattern-based complexity analysis of high-dimensional chaotic time series

The ordinal pattern-based complexity-entropy plane is a popular tool in nonlinear dynamics for distinguishing stochastic signals (noise) from deterministic chaos. Its performance, however, has mainly been demonstrated for time series from low-dimensional discrete or continuous dynamical systems. In order to evaluate the usefulness and power of the complexity-entropy (CE) plane approach for data representing high-dimensional chaotic dynamics, we applied this method to time series generated by the Lorenz-96 system, the generalized Hénon map, the Mackey-Glass equation, the Kuramoto-Sivashinsky equation, and to phase-randomized surrogates of these data. We find that both the high-dimensional deterministic time series and the stochastic surrogate data may be located in the same region of the complexity-entropy plane, and their representations show very similar behavior with varying lag and pattern lengths. Therefore, the classification of these data by means of their position in the CE plane can be challenging or even misleading, while surrogate data tests based on (entropy, complexity) yield significant results in most cases.

Rao-Burbea centroids applied to the statistical characterization of time series and images through ordinal patterns

Divergences or similarity measures between probability distributions have become a very useful tool for studying different aspects of statistical objects, such as time series, networks, and images. Notably, not every divergence provides identical results when applied to the same problem. Therefore, it seems convenient to have the widest possible set of divergences to be applied to the problems under study. Besides this choice, an essential step in the analysis of every statistical object is the mapping of each one of their representing values into an alphabet of symbols conveniently chosen. In this work, we choose the family of divergences known as the Burbea-Rao centroids (BRCs). For the mapping of the original time series into a symbolic sequence, we work with the ordinal pattern scheme. We apply our proposals to analyze simulated and real time series and to real textured images. The main conclusion of our work is that the best BRC, at least in the studied cases, is the Jensen-Shannon divergence, besides the fact that it verifies some interesting formal properties.

Markov-modulated model for landing flow dynamics: An ordinal analysis validation

Air transportation is a complex system characterized by a plethora of interactions at multiple temporal and spatial scales; as a consequence, even simple dynamics like sequencing aircraft for landing can lead to the appearance of emergent behaviors, which are both difficult to control and detrimental to operational efficiency. We propose a model, based on a modulated Markov jitter, to represent ordinal pattern properties of real landing operations in European airports. The parameters of the model are tuned by minimizing the distance between the probability distributions of ordinal patterns generated by the real and synthetic sequences, as estimated by the Permutation Jensen-Shannon Distance. We show that the correlation between consecutive hours in the landing flow changes between airports and that it can be interpreted as a metric of efficiency. We further compare the dynamics pre and post COVID-19, showing how this has changed beyond what can be attributed to a simple reduction of traffic. We finally draw some operational conclusions and discuss the applicability of these findings in a real operational environment.

Comparison of Information Criteria for Detection of Useful Signals in Noisy Environments

This paper considers the appearance of indications of useful acoustic signals in the signal/noise mixture. Various information characteristics (information entropy, Jensen-Shannon divergence, spectral information divergence and statistical complexity) are investigated in the context of solving this problem. Both time and frequency domains are studied for the calculation of information entropy. The effectiveness of statistical complexity is shown in comparison with other information metrics for different signal-to-noise ratios. Two different approaches for statistical complexity calculations are also compared. In addition, analytical formulas for complexity and disequilibrium are obtained using entropy variation in the case of signal spectral distribution. The connection between the statistical complexity criterion and the Neyman-Pearson approach for hypothesis testing is discussed. The effectiveness of the proposed approach is shown for different types of acoustic signals and noise models, including colored noises, and different signal-to-noise ratios, especially when the estimation of additional noise characteristics is impossible.

Complexity and Entropy in Physiological Signals (CEPS): Resonance Breathing Rate Assessed Using Measures of Fractal Dimension, Heart Rate Asymmetry and Permutation Entropy

BACKGROUND

As technology becomes more sophisticated, more accessible methods of interpretating Big Data become essential. We have continued to develop Complexity and Entropy in Physiological Signals (CEPS) as an open access MATLAB® GUI (graphical user interface) providing multiple methods for the modification and analysis of physiological data.

METHODS

To demonstrate the functionality of the software, data were collected from 44 healthy adults for a study investigating the effects on vagal tone of breathing paced at five different rates, as well as self-paced and un-paced. Five-minute 15-s recordings were used. Results were also compared with those from shorter segments of the data. Electrocardiogram (ECG), electrodermal activity (EDA) and Respiration (RSP) data were recorded. Particular attention was paid to COVID risk mitigation, and to parameter tuning for the CEPS measures. For comparison, data were processed using Kubios HRV, RR-APET and DynamicalSystems.jl software. We also compared findings for ECG RR interval (RRi) data resampled at 4 Hz (4R) or 10 Hz (10R), and non-resampled (noR). In total, we used around 190-220 measures from CEPS at various scales, depending on the analysis undertaken, with our investigation focused on three families of measures: 22 fractal dimension (FD) measures, 40 heart rate asymmetries or measures derived from Poincaré plots (HRA), and 8 measures based on permutation entropy (PE).

RESULTS

FDs for the RRi data differentiated strongly between breathing rates, whether data were resampled or not, increasing between 5 and 7 breaths per minute (BrPM). Largest effect sizes for RRi (4R and noR) differentiation between breathing rates were found for the PE-based measures. Measures that both differentiated well between breathing rates and were consistent across different RRi data lengths (1-5 min) included five PE-based (noR) and three FDs (4R). Of the top 12 measures with short-data values consistently within ± 5% of their values for the 5-min data, five were FDs, one was PE-based, and none were HRAs. Effect sizes were usually greater for CEPS measures than for those implemented in DynamicalSystems.jl.

CONCLUSION

The updated CEPS software enables visualisation and analysis of multichannel physiological data using a variety of established and recently introduced complexity entropy measures. Although equal resampling is theoretically important for FD estimation, it appears that FD measures may also be usefully applied to non-resampled data.

Goodness-of-fit test for stochastic processes using even empirical moments statistic

In this paper, we introduce a novel framework that allows efficient stochastic process discrimination. The underlying test statistic is based on even empirical moments and generalizes the time-averaged mean-squared displacement framework; the test is designed to allow goodness-of-fit statistical testing of processes with stationary increments and a finite-moment distribution. In particular, while our test statistic is based on a simple and intuitive idea, it enables efficient discrimination between finite- and infinite-moment processes even if the underlying laws are relatively close to each other. This claim is illustrated via an extensive simulation study, e.g., where we confront α-stable processes with stability index close to 2 with their standard Gaussian equivalents. For completeness, we also show how to embed our methodology into the real data analysis by studying the real metal price data.

Detection of cardiac arrhythmia patterns in ECG through H × C plane

The aim of this study is to formulate a new methodology based upon informational tools to detect patients with cardiac arrhythmias. As it is known, sudden death is the consequence of a final arrhythmia, and here lies the relevance of the efforts aimed at the early detection of arrhythmias. The information content in the time series from an electrocardiogram (ECG) signal is conveyed in the form of a probability distribution function, to compute the permutation entropy proposed by Bandt and Pompe. This selection was made seeking its remarkable conceptual simplicity, computational speed, and robustness to noise. In this work, two well-known databases were used, one containing normal sinus rhythms and another one containing arrhythmias, both from the MIT medical databank. For different values of embedding time delay τ, normalized permutation entropy and statistical complexity measure are computed to finally represent them on the horizontal and vertical axes, respectively, which define the causal plane H×C. To improve the results obtained in previous works, a feature set composed by these two magnitudes is built to train the following supervised machine learning algorithms: random forest (RF), support vector machine (SVM), and k nearest neighbors (kNN). To evaluate the performance of each classification technique, a 10-fold cross-validation scheme repeated 10 times was implemented. Finally, to select the best model, three quality parameters were computed, namely, accuracy, the area under the receiver operative characteristic (ROC) curve (AUC), and the F1-score. The results obtained show that the best classification model to detect the ECG coming from arrhythmic patients is RF. The values of the quality parameters were at the same levels reported in the available literature using a larger data set, thus supporting this proposal that uses a very small-sized feature space to train the model later used to classify. Summarizing, the attained results show the possibility to discriminate both groups of patients, with normal sinus rhythm or arrhythmic ECG, showing a promising efficiency in the definition of new markers for the detection of cardiovascular pathologies.

Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View

An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the "black-box" nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity-entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework.

Complexity of Brain Dynamics as a Correlate of Consciousness in Anaesthetized Monkeys

The use of anaesthesia is a fundamental tool in the investigation of consciousness. Anesthesia procedures allow to investigate different states of consciousness from sedation to deep anesthesia within controlled scenarios. In this study we use information quantifiers to measure the complexity of electrocorticogram recordings in monkeys. We apply these metrics to compare different stages of general anesthesia for evaluating consciousness in several anesthesia protocols. We find that the complexity of brain activity can be used as a correlate of consciousness. For two of the anaesthetics used, propofol and medetomidine, we find that the anaesthetised state is accompanied by a reduction in the complexity of brain activity. On the other hand we observe that use of ketamine produces an increase in complexity measurements. We relate this observation with increase activity within certain brain regions associated with the ketamine used doses. Our measurements indicate that complexity of brain activity is a good indicator for a general evaluation of different levels of consciousness awareness, both in anesthetized and non anesthetizes states.

A Detailed Systematic Review on Retinal Image Segmentation Methods

The separation of blood vessels in the retina is a major aspect in detecting ailment and is carried out by segregating the retinal blood vessels from the fundus images. Moreover, it helps to provide earlier therapy for deadly diseases and prevent further impacts due to diabetes and hypertension. Many reviews already exist for this problem, but those reviews have presented the analysis of a single framework. Hence, this article on retinal segmentation review has revealed distinct methodologies with diverse frameworks that are utilized for blood vessel separation. The novelty of this review research lies in finding the best neural network model by comparing its efficiency. For that, machine learning (ML) and deep learning (DL) were compared and have been reported as the best model. Moreover, different datasets were used to segment the retinal blood vessels. The execution of each approach is compared based on the performance metrics such as sensitivity, specificity, and accuracy using publically accessible datasets like STARE, DRIVE, ROSE, REFUGE, and CHASE. This article discloses the implementation capacity of distinct techniques implemented for each segmentation method. Finally, the finest accuracy of 98% and sensitivity of 96% were achieved for the technique of Convolution Neural Network with Ranking Support Vector Machine (CNN-rSVM). Moreover, this technique has utilized public datasets to verify efficiency. Hence, the overall review of this article has revealed a method for earlier diagnosis of diseases to deliver earlier therapy.

High-frequency oscillations in the ripple bands and amplitude information coding: Toward a biomarker of maximum entropy in the preictal signals

Intracranial electroencephalography (iEEG) can directly record local field potentials (LFPs) from a large set of neurons in the vicinity of the electrode. To search for possible epileptic biomarkers and to determine the epileptogenic zone that gives rise to seizures, we investigated the dynamics of basal and preictal signals. For this purpose, we explored the dynamics of the recorded time series for different frequency bands considering high-frequency oscillations (HFO) up to 240 Hz. We apply a Hilbert transform to study the amplitude and phase of the signals. The dynamics of the different frequency bands in the time causal entropy-complexity plane, H × C, is characterized by comparing the dynamical evolution of the basal and preictal time series. As the preictal states evolve closer to the time in which the epileptic seizure starts, the, H × C, dynamics changes for the higher frequency bands. The complexity evolves to very low values and the entropy becomes nearer to its maximal value. These quasi-stable states converge to equiprobable states when the entropy is maximal, and the complexity is zero. We could, therefore, speculate that in this case, it corresponds to the minimization of Gibbs free energy. In this case, the maximum entropy is equivalent to the principle of minimum consumption of resources in the system. We can interpret this as the nature of the system evolving temporally in the preictal state in such a way that the consumption of resources by the system is minimal for the amplitude in frequencies between 220-230 and 230-240 Hz.

Bet-hedging antimicrobial strategies in macrophage phagosome acidification drive the dynamics of Cryptococcus neoformans intracellular escape mechanisms

The fungus Cryptococcus neoformans is a major human pathogen with a remarkable intracellular survival strategy that includes exiting macrophages through non-lytic exocytosis (Vomocytosis) and transferring between macrophages (Dragotcytosis) by a mechanism that involves sequential events of non-lytic exocytosis and phagocytosis. Vomocytosis and Dragotcytosis are fungal driven processes, but their triggers are not understood. We hypothesized that the dynamics of Dragotcytosis could inherit the stochasticity of phagolysosome acidification and that Dragotcytosis was triggered by fungal cell stress. Consistent with this view, fungal cells involved in Dragotcytosis reside in phagolysosomes characterized by low pH and/or high oxidative stress. Using fluorescent microscopy, qPCR, live cell video microscopy, and fungal growth assays we found that the that mitigating pH or oxidative stress reduced Dragotcytosis frequency, whereas ROS susceptible mutants of C. neoformans underwent Dragotcytosis more frequently. Dragotcytosis initiation was linked to phagolysosomal pH, oxidative stresses, and macrophage polarization state. Dragotcytosis manifested stochastic dynamics thus paralleling the dynamics of phagosomal acidification, which correlated with the inhospitality of phagolysosomes in differently polarized macrophages. Hence, randomness in phagosomal acidification randomly created a population of inhospitable phagosomes where fungal cell stress triggered stochastic C. neoformans non-lytic exocytosis dynamics to escape a non-permissive intracellular macrophage environment.

Permutation Jensen-Shannon distance: A versatile and fast symbolic tool for complex time-series analysis

The main motivation of this paper is to introduce the permutation Jensen-Shannon distance, a symbolic tool able to quantify the degree of similarity between two arbitrary time series. This quantifier results from the fusion of two concepts, the Jensen-Shannon divergence and the encoding scheme based on the sequential ordering of the elements in the data series. The versatility and robustness of this ordinal symbolic distance for characterizing and discriminating different dynamics are illustrated through several numerical and experimental applications. Results obtained allow us to be optimistic about its usefulness in the field of complex time-series analysis. Moreover, thanks to its simplicity, low computational cost, wide applicability, and less susceptibility to outliers and artifacts, this ordinal measure can efficiently handle large amounts of data and help to tackle the current big data challenges.

Noninvasive Methods for Fault Detection and Isolation in Internal Combustion Engines Based on Chaos Analysis

The classic monitoring methods for detecting faults in automotive vehicles based on on-board diagnostics (OBD) are insufficient when diagnosing several mechanical failures. Other sensing techniques present drawbacks such as high invasiveness and limited physical range. The present work presents a fully noninvasive system for fault detection and isolation in internal combustion engines through sound signals processing. An acquisition system was developed, whose data are transmitted to a smartphone in which the signal is processed, and the user has access to the information. A study of the chaotic behavior of the vehicle was carried out, and the feasibility of using fractal dimensions as a tool to diagnose engine misfire and problems in the alternator belt was verified. An artificial neural network was used for fault classification using the fractal dimension data extracted from the sound of the engine. For comparison purposes, a strategy based on wavelet multiresolution analysis was also implemented. The proposed solution allows a diagnosis without having any contact with the vehicle, with low computational cost, without the need for installing sensors, and in real time. The system and method were validated through experimental tests, with a success rate of 99% for the faults under consideration.

Effect of gravity on synchronization of two coupled buoyancy-induced turbulent flames

We study the effect of gravity on the synchronization of two coupled buoyancy-induced turbulent flames by recurrence-based analysis and machine learning. A significant change from nearly complete synchronization in the near field to partial synchronization appears in the far field under low gravity. The synchronized state is gradually lost with increasing gravity level. These results are clearly identified from cross recurrence plots and symbolic recurrence plots and by reservoir computing.

Discriminating chaotic and stochastic time series using permutation entropy and artificial neural networks

Extracting relevant properties of empirical signals generated by nonlinear, stochastic, and high-dimensional systems is a challenge of complex systems research. Open questions are how to differentiate chaotic signals from stochastic ones, and how to quantify nonlinear and/or high-order temporal correlations. Here we propose a new technique to reliably address both problems. Our approach follows two steps: first, we train an artificial neural network (ANN) with flicker (colored) noise to predict the value of the parameter, [Formula: see text], that determines the strength of the correlation of the noise. To predict [Formula: see text] the ANN input features are a set of probabilities that are extracted from the time series by using symbolic ordinal analysis. Then, we input to the trained ANN the probabilities extracted from the time series of interest, and analyze the ANN output. We find that the [Formula: see text] value returned by the ANN is informative of the temporal correlations present in the time series. To distinguish between stochastic and chaotic signals, we exploit the fact that the difference between the permutation entropy (PE) of a given time series and the PE of flicker noise with the same [Formula: see text] parameter is small when the time series is stochastic, but it is large when the time series is chaotic. We validate our technique by analysing synthetic and empirical time series whose nature is well established. We also demonstrate the robustness of our approach with respect to the length of the time series and to the level of noise. We expect that our algorithm, which is freely available, will be very useful to the community.

ordpy: A Python package for data analysis with permutation entropy and ordinal network methods

Since Bandt and Pompe's seminal work, permutation entropy has been used in several applications and is now an essential tool for time series analysis. Beyond becoming a popular and successful technique, permutation entropy inspired a framework for mapping time series into symbolic sequences that triggered the development of many other tools, including an approach for creating networks from time series known as ordinal networks. Despite increasing popularity, the computational development of these methods is fragmented, and there were still no efforts focusing on creating a unified software package. Here, we present ordpy (http://github.com/arthurpessa/ordpy), a simple and open-source Python module that implements permutation entropy and several of the principal methods related to Bandt and Pompe's framework to analyze time series and two-dimensional data. In particular, ordpy implements permutation entropy, Tsallis and Rényi permutation entropies, complexity-entropy plane, complexity-entropy curves, missing ordinal patterns, ordinal networks, and missing ordinal transitions for one-dimensional (time series) and two-dimensional (images) data as well as their multiscale generalizations. We review some theoretical aspects of these tools and illustrate the use of ordpy by replicating several literature results.

Statistical complexity is maximized close to criticality in cortical dynamics

Complex systems are typically characterized as an intermediate situation between a complete regular structure and a random system. Brain signals can be studied as a striking example of such systems: cortical states can range from highly synchronous and ordered neuronal activity (with higher spiking variability) to desynchronized and disordered regimes (with lower spiking variability). It has been recently shown, by testing independent signatures of criticality, that a phase transition occurs in a cortical state of intermediate spiking variability. Here we use a symbolic information approach to show that, despite the monotonical increase of the Shannon entropy between ordered and disordered regimes, we can determine an intermediate state of maximum complexity based on the Jensen disequilibrium measure. More specifically, we show that statistical complexity is maximized close to criticality for cortical spiking data of urethane-anesthetized rats, as well as for a network model of excitable elements that presents a critical point of a nonequilibrium phase transition.

Complex network analysis of the gravity effect on premixed flames propagating in a Hele-Shaw cell

We study the effect of gravity on spatiotemporal flame front dynamics in a Hele-Shaw cell from the viewpoint of complex networks. The randomness in flame front dynamics significantly increases with the gravitational level when the normalized Rayleigh number R_{a} is negative. This is clearly identified by two network entropies: the flame front network entropy and the transition network entropy. The irregular formation of large-scale wrinkles driven by the Rayleigh-Taylor instability plays an important role in the formation of high-dimensional deterministic chaos at R_{a}<0, resulting in the increase in the randomness of flame front dynamics.

A generalized permutation entropy for noisy dynamics and random processes

Permutation entropy measures the complexity of a deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or simply permutations. Reasons for the increasing popularity of this entropy in time series analysis include that (i) it converges to the Kolmogorov-Sinai entropy of the underlying dynamics in the limit of ever longer permutations and (ii) its computation dispenses with generating and ad hoc partitions. However, permutation entropy diverges when the number of allowed permutations grows super-exponentially with their length, as happens when time series are output by dynamical systems with observational or dynamical noise or purely random processes. In this paper, we propose a generalized permutation entropy, belonging to the class of group entropies, that is finite in that situation, which is actually the one found in practice. The theoretical results are illustrated numerically by random processes with short- and long-term dependencies, as well as by noisy deterministic signals.

Learning dynamical systems in noise using convolutional neural networks

The problem of distinguishing deterministic chaos from non-chaotic dynamics has been an area of active research in time series analysis. Since noise contamination is unavoidable, it renders deterministic chaotic dynamics corrupted by noise to appear in close resemblance to stochastic dynamics. As a result, the problem of distinguishing noise-corrupted chaotic dynamics from randomness based on observations without access to the measurements of the state variables is difficult. We propose a new angle to tackle this problem by formulating it as a multi-class classification task. The task of classification involves allocating the observations/measurements to the unknown state variables in order to find the nature of these unobserved internal state variables. We employ signal and image processing based methods to characterize the different system dynamics. A deep learning technique using a state-of-the-art image classifier known as the Convolutional Neural Network (CNN) is designed to learn the dynamics. The time series are transformed into textured images of spectrogram and unthresholded recurrence plot (UTRP) for learning stochastic and deterministic chaotic dynamical systems in noise. We have designed a CNN that learns the dynamics of systems from the joint representation of the textured patterns from these images, thereby solving the problem as a pattern recognition task. The robustness and scalability of our approach is evaluated at different noise levels. Our approach demonstrates the advantage of applying the dynamical properties of chaotic systems in the form of joint representation of UTRP images along with spectrogram to improve learning dynamical systems in colored noise.

Macrophages use a bet-hedging strategy for antimicrobial activity in phagolysosomal acidification

Microbial ingestion by a macrophage results in the formation of an acidic phagolysosome but the host cell has no information on the pH susceptibility of the ingested organism. This poses a problem for the macrophage and raises the fundamental question of how the phagocytic cell optimizes the acidification process to prevail. We analyzed the dynamical distribution of phagolysosomal pH in murine and human macrophages that had ingested live or dead Cryptococcus neoformans cells, or inert beads. Phagolysosomal acidification produced a range of pH values that approximated normal distributions, but these differed from normality depending on ingested particle type. Analysis of the increments of pH reduction revealed no forbidden ordinal patterns, implying that the phagosomal acidification process was a stochastic dynamical system. Using simulation modeling, we determined that by stochastically acidifying a phagolysosome to a pH within the observed distribution, macrophages sacrificed a small amount of overall fitness to gain the benefit of reduced variation in fitness. Hence, chance in the final phagosomal pH introduces unpredictability to the outcome of the macrophage-microbe, which implies a bet-hedging strategy that benefits the macrophage. While bet hedging is common in biological systems at the organism level, our results show its use at the organelle and cellular level.

Contrasting chaotic with stochastic dynamics via ordinal transition networks

We introduce a representation space to contrast chaotic with stochastic dynamics. Following the complex network representation of a time series through ordinal pattern transitions, we propose to assign each system a position in a two-dimensional plane defined by the permutation entropy of the network (global network quantifier) and the minimum value of the permutation entropy of the nodes (local network quantifier). The numerical analysis of representative chaotic maps and stochastic systems shows that the proposed approach is able to distinguish linear from non-linear dynamical systems by different planar locations. Additionally, we show that this characterization is robust when observational noise is considered. Experimental applications allow us to validate the numerical findings and to conclude that this approach is useful in practical contexts.

Spatiotemporal dynamics of a buoyancy-driven turbulent fire

We numerically study the spatiotemporal dynamics and predictability of a buoyancy-driven turbulent fire. A significant transition from order to disorder structures can be observed from the mean degree in the spatial horizontal visibility graph. The gravitational term (baroclinic torque term) in the vorticity equation has a significant impact on the formation of the order (disorder) structure in the near field (far field). The entropy flow transport from temperature to flow velocity fluctuations is predominant near the interface between hot combustion products and ambient air. The transfer entropy is an important measure for determining the predictability of flow velocity fluctuations in the near field obtained by reservoir computing.

Parametric Jensen-Shannon Statistical Complexity and Its Applications on Full-Scale Compartment Fire Data

The order/disorder characteristics of a compartment fire are researched based on experimental data. From our analysis performed by new, pioneering methods, we claim that the parametric Jensen-Shannon complexity can be successfully used to detect unusual data, and that one can use it also as a means to perform relevant analysis of fire experiments. Thoroughly comparing the performance of different algorithms (known as permutation entropy and two-length permutation entropy) to extract the probability distribution is an essential step. We discuss some of the theoretical assumptions behind each step and stress that the role of the parameter is to fine-tune the results of the Jensen-Shannon statistical complexity. Note that the Jensen-Shannon statistical complexity is symmetric, while its parametric version displays a symmetric duality due to the a priori probabilities used.

Spatiotemporal dynamics of turbulent coaxial jet analyzed by symbolic information-theory quantifiers and complex-network approach

We numerically study the spatiotemporal dynamics of a turbulent coaxial jet in a model rocket engine combustor from the viewpoints of symbolic information-theory quantifiers and complex networks. The dynamic behavior of flow velocity undergoes a significant transition from a stochastic to chaotic state as the turbulent jet moves downstream. The small-world nature exists in the near field forming a stochastic state, whereas it disappears by the formation of a chaotic state in the far field. The dynamic behavior of hydrogen and oxygen concentrations in the far field also represents deterministic chaos. The simultaneous dynamic behavior with chaotic mixing forms the phase-synchronization state.

Markov modeling via ordinal partitions: An alternative paradigm for network-based time-series analysis

Mapping time series to complex networks to analyze observables has recently become popular, both at the theoretical and the practitioner's level. The intent is to use network metrics to characterize the dynamics of the underlying system. Applications cover a wide range of problems, from geoscientific measurements to biomedical data and financial time series. It has been observed that different dynamics can produce networks with distinct topological characteristics under a variety of time-series-to-network transforms that have been proposed in the literature. The direct connection, however, remains unclear. Here, we investigate a network transform based on computing statistics of ordinal permutations in short subsequences of the time series, the so-called ordinal partition network. We propose a Markovian framework that allows the interpretation of the network using ergodic-theoretic ideas and demonstrate, via numerical experiments on an ensemble of time series, that this viewpoint renders this technique especially well-suited to nonlinear chaotic signals. The aim is to test the mapping's faithfulness as a representation of the dynamics and the extent to which it retains information from the input data. First, we show that generating networks by counting patterns of increasing length is essentially a mechanism for approximating the analog of the Perron-Frobenius operator in a topologically equivalent higher-dimensional space to the original state space. Then, we illustrate a connection between the connectivity patterns of the networks generated by this mapping and indicators of dynamics such as the hierarchy of unstable periodic orbits embedded within a chaotic attractor. The input is a scalar observable and any projection of a multidimensional flow suffices for reconstruction of the essential dynamics. Additionally, we create a detailed guide for parameter tuning. We argue that there is no optimal value of the pattern length m, rather it admits a scaling region akin to traditional embedding practice. In contrast, the embedding lag and overlap between successive patterns can be chosen exactly in an optimal way. Our analysis illustrates the potential of this transform as a complementary toolkit to traditional time-series methods.

Unraveling the decay of the number of unobserved ordinal patterns in noisy chaotic dynamics

In this paper, we introduce a model to describe the decay of the number of unobserved ordinal patterns as a function of the time series length in noisy chaotic dynamics. More precisely, we show that a stretched exponential model fits the decay of the number of unobserved ordinal patterns for both discrete and continuous chaotic systems contaminated with observational noise, independently of the noise level and the sampling time. Numerical simulations, obtained from the logistic map and the x coordinate of the Lorenz system, both operating in a totally chaotic dynamics were used as test beds. In addition, we contrast our results with those obtained from pure stochastic dynamics. The fitting parameters, namely, the stretching exponent and the characteristic decay rate, are used to distinguish whether the dynamical nature of the data sequence is stochastic or chaotic. Finally, the analysis of experimental records associated with the hyperchaotic pulsations of an optoelectronic oscillator allows us to illustrate the applicability of the proposed approach in a practical context.

Feature Extraction of Ship-Radiated Noise Based on Intrinsic Time-Scale Decomposition and a Statistical Complexity Measure

Extracting effective features from ship-radiated noise is an important way to improve the detection and recognition performance of passive sonar. Complexity features of ship-radiated noise have attracted increasing amounts of attention. However, the traditional definition of complexity based on entropy (information stored in the system) is not accurate. To this end, a new statistical complexity measure is proposed in this paper based on spectrum entropy and disequilibrium. Since the spectrum features are unique to the class of the ship, our method can distinguish different ships according to their location in the two-dimensional plane composed of complexity and spectrum entropy (CSEP). To weaken the influence of ocean ambient noise, the intrinsic time-scale decomposition (ITD) is applied to preprocess the data in this study. The effectiveness of the proposed method is validated through a classification experiment of four types of marine vessels. The recognition rate of the ITD-CSEP methodology achieved 94%, which is much higher than that of traditional feature extraction methods. Moreover, the ITD-CSEP is fast and parameter free. Hence, the method can be applied in the real time processing practical applications.

Symbolic diffusion entropy rate of chaotic time series as a surrogate measure for the largest Lyapunov exponent

Existing methods for estimating the largest Lyapunov exponent from a time series rely on the rate of separation of initially nearby trajectories reconstructed from the time series in phase space. According to Ueda, chaotic dynamical behavior is viewed as a manifestation of random transitions between unstable periodic orbits in a chaotic attractor, which are triggered by perturbations due to experimental observation or the roundoff error characteristic of the computing machine, and consequently consists of a sequence of piecewise deterministic processes instead of an entirely deterministic process. Chaotic trajectories might have no physical reality. Here, we propose a mathematical method for estimating a surrogate measure for the largest Lyapunov exponent on the basis of the random diffusion of the symbols generated from a time series in a chaotic attractor, without resorting to initially nearby trajectories. We apply the proposed method to numerical time series generated by chaotic flow models and verify its validity.

Early detection of thermoacoustic combustion oscillations using a methodology combining statistical complexity and machine learning

We conduct an experimental study on early detection of thermoacoustic combustion oscillations using a method combining statistical complexity and machine learning, including the characterization of intermittent combustion oscillations. Abrupt switching from aperiodic small-amplitude oscillations to periodic large-amplitude oscillations and vice versa appears in pressure fluctuations. The dynamic behavior of aperiodic small-amplitude pressure fluctuations represents chaos. The complexity-entropy causality plane effectively captures the subtle changes in the combustion state during a transition to well-developed combustion oscillations. The feature space of the complexity-entropy causality plane, which is obtained by a support vector machine, has potential use for detecting a precursor of combustion oscillations.

Spiky electric and magnetic field structures in flux rope experiments

Magnetic flux ropes are structures that are common in the corona of the sun and presumably all stars. They can be thought of as the building blocks of solar structures. They have been observed in Earth's magnetotail and near Mars and Venus. When multiple flux ropes are present magnetic field line reconnection, which converts magnetic energy to other forms, can occur when they collide. The structure of multiple magnetic ropes, the interactions between multiple ropes, and their topological properties such as helicity and writhing have been studied theoretically and in laboratory experiments. Here, we report on spiky potential and magnetic fields associated with the ropes. We show that the potential structures are chaotic for a range of their temporal half-widths and the probability density function (PDF) of their widths resembles the statistical distribution of crumpled paper. The spatial structure of the magnetic spikes is revealed using a correlation counting method. Computer simulation suggests that the potential structures are the nonlinear end result of an instability involving relative drift between ions and electrons.

On the predictability of infectious disease outbreaks

Infectious disease outbreaks recapitulate biology: they emerge from the multi-level interaction of hosts, pathogens, and environment. Therefore, outbreak forecasting requires an integrative approach to modeling. While specific components of outbreaks are predictable, it remains unclear whether fundamental limits to outbreak prediction exist. Here, adopting permutation entropy as a model independent measure of predictability, we study the predictability of a diverse collection of outbreaks and identify a fundamental entropy barrier for disease time series forecasting. However, this barrier is often beyond the time scale of single outbreaks, implying prediction is likely to succeed. We show that forecast horizons vary by disease and that both shifting model structures and social network heterogeneity are likely mechanisms for differences in predictability. Our results highlight the importance of embracing dynamic modeling approaches, suggest challenges for performing model selection across long time series, and may relate more broadly to the predictability of complex adaptive systems.

Detection of time reversibility in time series by ordinal patterns analysis

Time irreversibility is a common signature of nonlinear processes and a fundamental property of non-equilibrium systems driven by non-conservative forces. A time series is said to be reversible if its statistical properties are invariant regardless of the direction of time. Here, we propose the Time Reversibility from Ordinal Patterns method (TiROP) to assess time-reversibility from an observed finite time series. TiROP captures the information of scalar observations in time forward as well as its time-reversed counterpart by means of ordinal patterns. The method compares both underlying information contents by quantifying its (dis)-similarity via the Jensen-Shannon divergence. The statistic is contrasted with a population of divergences coming from a set of surrogates to unveil the temporal nature and its involved time scales. We tested TiROP in different synthetic and real, linear, and non-linear time series, juxtaposed with results from the classical Ramsey's time reversibility test. Our results depict a novel, fast-computation, and fully data-driven methodology to assess time-reversibility with no further assumptions over data. This approach adds new insights into the current non-linear analysis techniques and also could shed light on determining new physiological biomarkers of high reliability and computational efficiency.

Multiscale permutation Rényi entropy and its application for EEG signals

There is considerable interest in analyzing the complexity of electroencephalography (EEG) signals. However, some traditional complexity measure algorithms only quantify the complexities of signals, but cannot discriminate different signals very well. To analyze the complexity of epileptic EEG signals better, a new multiscale permutation Rényi entropy (MPEr) algorithm is proposed. In this algorithm, the coarse-grained procedure is introduced by using weighting-averaging method, and the weighted factors are determined by analyzing nonlinear signals. We apply the new algorithm to analyze epileptic EEG signals. The experimental results show that MPEr algorithm has good performance for discriminating different EEG signals. Compared with permutation Rényi entropy (PEr) and multiscale permutation entropy (MPE), MPEr distinguishes different EEG signals successfully. The proposed MPEr algorithm is effective and has good applications prospects in EEG signals analysis.

Causal Shannon-Fisher Characterization of Motor/Imagery Movements in EEG

The electroencephalogram (EEG) is an electrophysiological monitoring method that allows us to glimpse the electrical activity of the brain. Neural oscillations patterns are perhaps the best salient feature of EEG as they are rhythmic activities of the brain that can be generated by interactions across neurons. Large-scale oscillations can be measured by EEG as the different oscillation patterns reflected within the different frequency bands, and can provide us with new insights into brain functions. In order to understand how information about the rhythmic activity of the brain during visuomotor/imagined cognitive tasks is encoded in the brain we precisely quantify the different features of the oscillatory patterns considering the Shannon–Fisher plane H×F. This allows us to distinguish the dynamics of rhythmic activities of the brain showing that the Beta band facilitate information transmission during visuomotor/imagined tasks.

Assessing sustainability in North America's ecosystems using criticality and information theory

Sustainability is a key concept in economic and policy debates. Nevertheless, it is usually treated only in a qualitative way and has eluded quantitative analysis. Here, we propose a sustainability index based on the premise that sustainable systems do not lose or gain Fisher Information over time. We test this approach using time series data from the AmeriFlux network that measures ecosystem respiration, water and energy fluxes in order to elucidate two key sustainability features: ecosystem health and stability. A novel definition of ecosystem health is developed based on the concept of criticality, which implies that if a system's fluctuations are scale invariant then the system is in a balance between robustness and adaptability. We define ecosystem stability by taking an information theory approach that measures its entropy and Fisher information. Analysis of the Ameriflux consortium big data set of ecosystem respiration time series is contrasted with land condition data. In general we find a good agreement between the sustainability index and land condition data. However, we acknowledge that the results are a preliminary test of the approach and further verification will require a multi-signal analysis. For example, high values of the sustainability index for some croplands are counter-intuitive and we interpret these results as ecosystems maintained in artificial health due to continuous human-induced inflows of matter and energy in the form of soil nutrients and control of competition, pests and disease.

Analysis of ischaemic crisis using the informational causal entropy-complexity plane

In the present work, an ischaemic process, mainly focused on the reperfusion stage, is studied using the informational causal entropy-complexity plane. Ischaemic wall behavior under this condition was analyzed through wall thickness and ventricular pressure variations, acquired during an obstructive flow maneuver performed on left coronary arteries of surgically instrumented animals. Basically, the induction of ischaemia depends on the temporary occlusion of left circumflex coronary artery (which supplies blood to the posterior left ventricular wall) that lasts for a few seconds. Normal perfusion of the wall was then reestablished while the anterior ventricular wall remained adequately perfused during the entire maneuver. The obtained results showed that system dynamics could be effectively described by entropy-complexity loops, in both abnormally and well perfused walls. These results could contribute to making an objective indicator of the recovery heart tissues after an ischaemic process, in a way to quantify the restoration of myocardial behavior after the supply of oxygen to the ventricular wall was suppressed for a brief period.

Bandt-Pompe symbolization dynamics for time series with tied values: A data-driven approach

In 2002, Bandt and Pompe [Phys. Rev. Lett. 88, 174102 (2002)] introduced a successfully symbolic encoding scheme based on the ordinal relation between the amplitude of neighboring values of a given data sequence, from which the permutation entropy can be evaluated. Equalities in the analyzed sequence, for example, repeated equal values, deserve special attention and treatment as was shown recently by Zunino and co-workers [Phys. Lett. A 381, 1883 (2017)]. A significant number of equal values can give rise to false conclusions regarding the underlying temporal structures in practical contexts. In the present contribution, we review the different existing methodologies for treating time series with tied values by classifying them according to their different strategies. In addition, a novel data-driven imputation is presented that proves to outperform the existing methodologies and avoid the false conclusions pointed by Zunino and co-workers.

The Chaotic Long-term X-ray Variability of 4U 1705-44

The low-mass X-ray binary 4U1705-44 exhibits dramatic long-term X-ray time variability with a timescale of several hundred days. The All-Sky Monitor (ASM) aboard the Rossi X-ray Timing Explorer (RXTE) and the Japanese Monitor of All-sky X-ray Image (MAXI) aboard the International Space Station together have continuously observed the source from December 1995 through May 2014. The combined ASM-MAXI data provide a continuous time series over fifty times the length of the timescale of interest. Topological analysis can help us identify fingerprints in the phase-space of a system unique to its equations of motion. The Birman-Williams theorem postulates that if such fingerprints are the same between two systems, then their equations of motion must be closely related. The phase-space embedding of the source light curve shows a strong resemblance to the double-welled nonlinear Duffing oscillator. We explore a range of parameters for which the Duffing oscillator closely mirrors the time evolution of 4U1705-44. We extract low period, unstable periodic orbits from the 4U1705-44 and Duffing time series and compare their topological information. The Duffing and 4U1705-44 topological properties are identical, providing strong evidence that they share the same underlying template. This suggests that we can look to the Duffing equation to help guide the development of a physical model to describe the long-term X-ray variability of this and other similarly behaved X-ray binary systems.

Rhythmic activities of the brain: Quantifying the high complexity of beta and gamma oscillations during visuomotor tasks

Electroencephalography (EEG) signals depict the electrical activity that takes place at the surface of the brain and provide an important tool for understanding a variety of cognitive processes. The EEG is the product of synchronized activity of the brain, and variations in EEG oscillations patterns reflect the underlying changes in neuronal synchrony. Our aim is to characterize the complexity of the EEG rhythmic oscillations bands when the subjects perform a visuomotor or imagined cognitive tasks (imagined movement), providing a causal mapping of the dynamical rhythmic activities of the brain as a measure of attentional investment. We estimate the intrinsic correlational structure of the signals within the causality entropy-complexity plane H×C, where the enhanced complexity in the gamma 1, gamma 2, and beta 1 bands allows us to distinguish motor-visual memory tasks from control conditions. We identify the dynamics of the gamma 1, gamma 2, and beta 1 rhythmic oscillations within the zone of a chaotic dissipative behavior, whereas in contrast the beta 2 band shows a much higher level of entropy and a significant low level of complexity that correspond to a non-invertible cubic map. Our findings enhance the importance of the gamma band during attention in perceptual feature binding during the visuomotor/imagery tasks.

Dynamic behavior of combustion instability in a cylindrical combustor with an off-center installed coaxial injector

We have intensively studied the dynamic behavior of combustion instability in a cylindrical combustor with an off-center installed coaxial injector. The most interesting discovery in this study is the appearance of a deterministic chaos in a transition from a dynamically stable state to well-developed high-frequency thermoacoustic combustion oscillations with increasing the volume flow rate of nitrogen with which oxygen is diluted. The presence of deterministic chaos is reasonably identified by considering an extended version of the Sugihara-May algorithm [G. Sugihara and R. May, Nature 344, 734 (1990)] as a local predictor and the multiscale complexity-entropy causality plane based on statistical complexity.