Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning

Time series analysis comprises a wide repertoire of methods for extracting information from data sets. Despite great advances in time series analysis, identifying and quantifying the strength of nonlinear temporal correlations remain a challenge. We have recently proposed a new method based on training a machine learning algorithm to predict the temporal correlation parameter, α, of flicker noise (FN) time series. The algorithm is trained using as input features the probabilities of ordinal patterns computed from FN time series, xαFN(t), generated with different values of α. Then, the ordinal probabilities computed from the time series of interest, x(t), are used as input features to the trained algorithm and that returns a value, αe, that contains meaningful information about the temporal correlations present in x(t). We have also shown that the difference, Ω, of the permutation entropy (PE) of the time series of interest, x(t), and the PE of a FN time series generated with α=αe, xαeFN(t), allows the identification of the underlying determinism in x(t). Here, we apply our methodology to different datasets and analyze how αe and Ω correlate with well-known quantifiers of chaos and complexity. We also discuss the limitations for identifying determinism in highly chaotic time series and in periodic time series contaminated by noise. The open source algorithm is available on Github.

Time-Delay Identification Using Multiscale Ordinal Quantifiers

Time-delayed interactions naturally appear in a multitude of real-world systems due to the finite propagation speed of physical quantities. Often, the time scales of the interactions are unknown to an external observer and need to be inferred from time series of observed data. We explore, in this work, the properties of several ordinal-based quantifiers for the identification of time-delays from time series. To that end, we generate artificial time series of stochastic and deterministic time-delay models. We find that the presence of a nonlinearity in the generating model has consequences for the distribution of ordinal patterns and, consequently, on the delay-identification qualities of the quantifiers. Here, we put forward a novel ordinal-based quantifier that is particularly sensitive to nonlinearities in the generating model and compare it with previously-defined quantifiers. We conclude from our analysis on artificially generated data that the proper identification of the presence of a time-delay and its precise value from time series benefits from the complementary use of ordinal-based quantifiers and the standard autocorrelation function. We further validate these tools with a practical example on real-world data originating from the North Atlantic Oscillation weather phenomenon.

Chronic apnea during REM sleep increases arterial pressure and sympathetic modulation in rats

STUDY OBJECTIVES

Obstructive sleep apnea can induce hypertension. Apneas in REM may be particularly problematic: they are independently associated with hypertension. We examined the role of sleep stage and awakening on acute cardiovascular responses to apnea. In addition, we measured cardiovascular and sympathetic changes induced by chronic sleep apnea in REM sleep.

METHODS

We used rats with tracheal balloons and electroencephalogram and electromyogram electrodes to induce obstructive apnea during wakefulness and sleep. We measured the electrocardiogram and arterial pressure by telemetry and breathing effort with a thoracic balloon.

RESULTS

Apneas induced during wakefulness caused a pressor response, intense bradycardia, and breathing effort. On termination of apnea, arterial pressure, heart rate, and breathing effort returned to basal levels within 10 s. Responses to apnea were strongly blunted when apneas were made in sleep. Post-apnea changes were also blunted when rats did not awake from apnea. Chronic sleep apnea (15 days of apnea during REM sleep, 8 h/day, 13.8 ± 2 apneas/h, average duration 12 ± 0.7 s) reduced sleep time, increased awake arterial pressure from 111 ± 6 to 118 ± 5 mmHg (p < 0.05) and increased a marker for sympathetic activity. Chronic apnea failed to change spontaneous baroreceptor sensitivity.

CONCLUSION

Our results suggest that sleep blunts the diving-like response induced by apnea and that acute post-apnea changes depend on awakening. In addition, our data confirm that 2 weeks of apnea during REM causes sleep disruption and increases blood pressure and sympathetic activity.

Selection of Embedding Dimension and Delay Time in Phase Space Reconstruction via Symbolic Dynamics

The modeling and prediction of chaotic time series require proper reconstruction of the state space from the available data in order to successfully estimate invariant properties of the embedded attractor. Thus, one must choose appropriate time delay τ∗ and embedding dimension p for phase space reconstruction. The value of τ∗ can be estimated from the Mutual Information, but this method is rather cumbersome computationally. Additionally, some researchers have recommended that τ∗ should be chosen to be dependent on the embedding dimension p by means of an appropriate value for the time delay τw=(p−1)τ∗, which is the optimal time delay for independence of the time series. The C-C method, based on Correlation Integral, is a method simpler than Mutual Information and has been proposed to select optimally τw and τ∗. In this paper, we suggest a simple method for estimating τ∗ and τw based on symbolic analysis and symbolic entropy. As in the C-C method, τ∗ is estimated as the first local optimal time delay and τw as the time delay for independence of the time series. The method is applied to several chaotic time series that are the base of comparison for several techniques. The numerical simulations for these systems verify that the proposed symbolic-based method is useful for practitioners and, according to the studied models, has a better performance than the C-C method for the choice of the time delay and embedding dimension. In addition, the method is applied to EEG data in order to study and compare some dynamic characteristics of brain activity under epileptic episodes

Neuronal Transmission of Subthreshold Periodic Stimuli Via Symbolic Spike Patterns

We study how sensory neurons detect and transmit a weak external stimulus. We use the FitzHugh-Nagumo model to simulate the neuronal activity. We consider a sub-threshold stimulus, i.e., the stimulus is below the threshold needed for triggering action potentials (spikes). However, in the presence of noise the neuron that perceives the stimulus fires a sequence of action potentials (a spike train) that carries the stimulus' information. To yield light on how the stimulus' information can be encoded and transmitted, we consider the simplest case of two coupled neurons, such that one neuron (referred to as neuron 1) perceives a subthreshold periodic signal but the second neuron (neuron 2) does not perceive the signal. We show that, for appropriate coupling and noise strengths, both neurons fire spike trains that have symbolic patterns (defined by the temporal structure of the inter-spike intervals), whose frequencies of occurrence depend on the signal's amplitude and period, and are similar for both neurons. In this way, the signal information encoded in the spike train of neuron 1 propagates to the spike train of neuron 2. Our results suggest that sensory neurons can exploit the presence of neural noise to fire spike trains where the information of a subthreshold stimulus is encoded in over expressed and/or in less expressed symbolic patterns.

Sympatho-Vagal Dysfunction in Patients with End-Stage Lung Disease Awaiting Lung Transplantation

Although the literature demonstrates that cardiac autonomic control (CAC) might be impaired in patients with chronic pulmonary diseases, the interplay between CAC and disease severity in end-stage lung disease has not been studied yet. We investigated the effects of end-stage lung disease on CAC through the analysis of heart rate variability (HRV) among patients awaiting lung transplantation. Forty-nine patients on the waiting list for lung transplantation (LTx; 19 men, age 38 ± 15 years) and 49 healthy non-smoking controls (HC; 22 men, age 40 ± 16 years) were enrolled in a case-control study at Policlinico Hospital in Milan, Italy. LTx patients were divided into two groups, according to disease severity evaluated by the Lung Allocation Score (LAS). To assess CAC, electrocardiogram (ECG) and respiration were recorded at rest for 10 min in supine position and for 10 min during active standing. Spectral analysis identified low and high frequencies (LF, sympathetic, and HF, vagal). Symbolic analysis identified three patterns, i.e., 0V% (sympathetic) and 2UV% and 2LV% (vagal). Compared to HCs, LTx patients showed higher markers of sympathetic modulation and lower markers of vagal modulation. However, more severely affected LTx patients, compared to less severely affected ones, showed an autonomic profile characterized by loss of sympathetic modulation and predominant vagal modulation. This pattern can be due to a loss of sympathetic rhythmic oscillation and a subsequent prevalent respiratory modulation of heart rate in severely affected patients.

Permutation Entropy as a Measure of Information Gain/Loss in the Different Symbolic Descriptions of Financial Data

Financial markets give a large number of trading opportunities. However, over-complicated systems make it very difficult to be effectively used by decision-makers. Volatility and noise present in the markets evoke a need to simplify the market picture derived for the decision-makers. Symbolic representation fits in this concept and greatly reduces data complexity. However, at the same time, some information from the market is lost. Our motivation is to answer the question: What is the impact of introducing different data representation on the overall amount of information derived for the decision-maker? We concentrate on the possibility of using entropy as a measure of the information gain/loss for the financial data, and as a basic form, we assume permutation entropy with later modifications. We investigate different symbolic representations and compare them with classical data representation in terms of entropy. The real-world data covering the time span of 10 years are used in the experiments. The results and the statistical verification show that extending the symbolic description of the time series does not affect the permutation entropy values.

Time Series Complexities and Their Relationship to Forecasting Performance

Entropy is a key concept in the characterization of uncertainty for any given signal, and its extensions such as Spectral Entropy and Permutation Entropy. They have been used to measure the complexity of time series. However, these measures are subject to the discretization employed to study the states of the system, and identifying the relationship between complexity measures and the expected performance of the four selected forecasting methods that participate in the M4 Competition. This relationship allows the decision, in advance, of which algorithm is adequate. Therefore, in this paper, we found the relationships between entropy-based complexity framework and the forecasting error of four selected methods (Smyl, Theta, ARIMA, and ETS). Moreover, we present a framework extension based on the Emergence, Self-Organization, and Complexity paradigm. The experimentation with both synthetic and M4 Competition time series show that the feature space induced by complexities, visually constrains the forecasting method performance to specific regions; where the logarithm of its metric error is poorer, the Complexity based on the emergence and self-organization is maximal.

Symbolic Recurrence Analysis of RR Interval to Detect Atrial Fibrillation

Atrial fibrillation (AF) is a sustained cardiac arrhythmia associated with stroke, heart failure, and related health conditions. Though easily diagnosed upon presentation in a clinical setting, the transient and/or intermittent emergence of AF episodes present diagnostic and clinical monitoring challenges that would ideally be met with automated ambulatory monitoring and detection. Current approaches to address these needs, commonly available both in smartphone applications and dedicated technologies, combine electrocardiogram (ECG) sensors with predictive algorithms to detect AF. These methods typically require extensive preprocessing, preliminary signal analysis, and the integration of a wide and complex array of features for the detection of AF events, and are consequently vulnerable to over-fitting. In this paper, we introduce the application of symbolic recurrence quantification analysis (SRQA) for the study of ECG signals and detection of AF events, which requires minimal pre-processing and allows the construction of highly accurate predictive algorithms from relatively few features. In addition, this approach is robust against commonly-encountered signal processing challenges that are expected in ambulatory monitoring contexts, including noisy and non-stationary data. We demonstrate the application of this method to yield a highly accurate predictive algorithm, which at optimal threshold values is 97.9% sensitive, 97.6% specific, and 97.7% accurate in classifying AF signals. To confirm the robust generalizability of this approach, we further evaluated its performance in the implementation of a 10-fold cross-validation paradigm, yielding 97.4% accuracy. In sum, these findings emphasize the robust utility of SRQA for the analysis of ECG signals and detection of AF. To the best of our knowledge, the proposed model is the first to incorporate symbolic analysis for AF beat detection.

Algorithmics, Possibilities and Limits of Ordinal Pattern Based Entropies

The study of nonlinear and possibly chaotic time-dependent systems involves long-term data acquisition or high sample rates. The resulting big data is valuable in order to provide useful insights into long-term dynamics. However, efficient and robust algorithms are required that can analyze long time series without decomposing the data into smaller series. Here symbolic-based analysis techniques that regard the dependence of data points are of some special interest. Such techniques are often prone to capacity or, on the contrary, to undersampling problems if the chosen parameters are too large. In this paper we present and apply algorithms of the relatively new ordinal symbolic approach. These algorithms use overlapping information and binary number representation, whilst being fast in the sense of algorithmic complexity, and allow, to the best of our knowledge, larger parameters than comparable methods currently used. We exploit the achieved large parameter range to investigate the limits of entropy measures based on ordinal symbolics. Moreover, we discuss data simulations from this viewpoint.

Cardiac and Peripheral Autonomic Responses to Orthostatic Stress During Transcutaneous Vagus Nerve Stimulation in Healthy Subjects

Previous studies showed that transcutaneous vagus nerve stimulation (tVNS) modulates the autonomic nervous system (ANS) in resting condition. However, the autonomic regulation in response to an orthostatic challenge during tVNS in healthy subjects remains unknown. We tested the hypothesis that tVNS reduces heart rate (HR) and alters the responsivity of ANS to orthostatic stress in healthy subjects. In a randomized and cross-over trial, thirteen healthy subjects underwent two experimental sessions on different days: (1) tVNS and (2) control. Using a tVNS device, an auricular electrode was placed on the left cymba conchae of the external ear; an electric current with a pulse frequency of 25 Hz and amplitude between 1 and 6 mA was applied. For the assessment of ANS, the beat-to-beat HR and systolic arterial pressure (SAP) were analyzed using linear and nonlinear approaches during clinostatic and orthostatic conditions. In clinostatic conditions, tVNS reduced HR (p < 0.01), SAP variability (p < 0.01), and cardiac and peripheral sympathetic modulation (p < 0.01). The responsivity of the peripheral sympathetic modulation to orthostatic stress during tVNS was significantly higher when compared to the control session (p = 0.03). In conclusion, tVNS reduces the HR and affects cardiac and peripheral autonomic control and increases the responses of peripheral autonomic control to orthostatic stress in healthy subjects.

Automatically Selecting a Suitable Integration Scheme for Systems of Differential Equations in Neuron Models

On the level of the spiking activity, the integrate-and-fire neuron is one of the most commonly used descriptions of neural activity. A multitude of variants has been proposed to cope with the huge diversity of behaviors observed in biological nerve cells. The main appeal of this class of model is that it can be defined in terms of a hybrid model, where a set of mathematical equations describes the sub-threshold dynamics of the membrane potential and the generation of action potentials is often only added algorithmically without the shape of spikes being part of the equations. In contrast to more detailed biophysical models, this simple description of neuron models allows the routine simulation of large biological neuronal networks on standard hardware widely available in most laboratories these days. The time evolution of the relevant state variables is usually defined by a small set of ordinary differential equations (ODEs). A small number of evolution schemes for the corresponding systems of ODEs are commonly used for many neuron models, and form the basis of the neuron model implementations built into commonly used simulators like Brian, NEST and NEURON. However, an often neglected problem is that the implemented evolution schemes are only rarely selected through a structured process based on numerical criteria. This practice cannot guarantee accurate and stable solutions for the equations and the actual quality of the solution depends largely on the parametrization of the model. In this article, we give an overview of typical equations and state descriptions for the dynamics of the relevant variables in integrate-and-fire models. We then describe a formal mathematical process to automate the design or selection of a suitable evolution scheme for this large class of models. Finally, we present the reference implementation of our symbolic analysis toolbox for ODEs that can guide modelers during the implementation of custom neuron models.

Reliability of Static and Dynamic Network Metrics in the Resting-State: A MEG-Beamformed Connectivity Analysis

The resting activity of the brain can be described by so-called intrinsic connectivity networks (ICNs), which consist of spatially and temporally distributed, but functionally connected, nodes. The coordinated activity of the resting state can be explored via magnetoencephalography (MEG) by studying frequency-dependent functional brain networks at the source level. Although many algorithms for the analysis of brain connectivity have been proposed, the reliability of network metrics derived from both static and dynamic functional connectivity is still unknown. This is a particular problem for studies of associations between ICN metrics and personality variables or other traits, and for studies of differences between patient and control groups, which both depend critically on the reliability of the metrics used. A detailed investigation of the reliability of metrics derived from resting-state MEG repeat scans is therefore a prerequisite for the development of connectomic biomarkers. Here, we first estimated both static (SFC) and dynamic functional connectivity (DFC) after beamforming source reconstruction using the imaginary part of the phase locking index (iPLV) and the correlation of the amplitude envelope (CorEnv). Using our approach, functional network microstates (FCμstates) were derived from the DFC and chronnectomics were computed from the evolution of FCμstates across experimental time. In both temporal scales, the reliability of network metrics (SFC), the FCμstates and the related chronnectomics were evaluated for every frequency band. Chronnectomic statistics and FCμstates were generally more reliable than node-wise static network metrics. CorEnv-based network metrics were more reproducible at the static approach. The reliability of chronnectomics have been evaluated also in a second dataset. This study encourages the analysis of MEG resting-state via DFC.

Mining Time-Resolved Functional Brain Graphs to an EEG-Based Chronnectomic Brain Aged Index (CBAI)

The brain at rest consists of spatially and temporal distributed but functionally connected regions that called intrinsic connectivity networks (ICNs). Resting state electroencephalography (rs-EEG) is a way to characterize brain networks without confounds associated with task EEG such as task difficulty and performance. A novel framework of how to study dynamic functional connectivity under the notion of functional connectivity microstates (FCμstates) and symbolic dynamics is further discussed. Furthermore, we introduced a way to construct a single integrated dynamic functional connectivity graph (IDFCG) that preserves both the strength of the connections between every pair of sensors but also the type of dominant intrinsic coupling modes (DICM). The whole methodology is demonstrated in a significant and unexplored task for EEG which is the definition of an objective Chronnectomic Brain Aged index (CBAI) extracted from resting-state data (N = 94 subjects) with both eyes-open and eyes-closed conditions. Novel features have been defined based on symbolic dynamics and the notion of DICM and FCμstates. The transition rate of FCμstates, the symbolic dynamics based on the evolution of FCμstates (the Markovian Entropy, the complexity index), the probability distribution of DICM, the novel Flexibility Index that captures the dynamic reconfiguration of DICM per pair of EEG sensors and the relative signal power constitute a valuable pool of features that can build the proposed CBAI. Here we applied a feature selection technique and Extreme Learning Machine (ELM) classifier to discriminate young adults from middle-aged and a Support Vector Regressor to build a linear model of the actual age based on EEG-based spatio-temporal features. The most significant type of features for both prediction of age and discrimination of young vs. adults age groups was the dynamic reconfiguration of dominant coupling modes derived from a subset of EEG sensor pairs. Specifically, our results revealed a very high prediction of age for eyes-open (R² = 0.60; y = 0.79x + 8.03) and lower for eyes-closed (R² = 0.48; y = 0.71x + 10.91) while we succeeded to correctly classify young vs. middle-age group with 97.8% accuracy in eyes-open and 87.2% for eyes-closed. Our results were reproduced also in a second dataset for further external validation of the whole analysis. The proposed methodology proved valuable for the characterization of the intrinsic properties of dynamic functional connectivity through the age untangling developmental differences using EEG resting-state recordings.

Cardiac autonomic control and complexity during sleep are preserved after chronic sleep restriction in healthy subjects

Acute sleep deprivation (SD) alters cardiovascular autonomic control (CAC) and is associated with an increased risk of cardiovascular disorders. However, the effects of partial SD on CAC are unclear. Thus, we aimed to investigate the effects of partial SD on CAC during sleep. We randomized seventeen healthy subjects to a restriction group (RES, n = 8, subjects slept two-thirds of normal sleep time based on individual habitual sleep duration for 8 days and 8 nights) or a Control group (CON, n = 9, subjects were allowed to sleep their usual sleep time). Attended polysomnographic (PSG) studies were performed every night; a subset of them was selected for the analysis at baseline (day 3-D3), the first night after sleep restriction (day 5-D5), at the end of sleep restriction period (day 11-D11), and at the end of recovery phase (day 14-D14). We extracted electrocardiogram (ECG) and respiration from the PSG and divided into wakefulness (W), nonrapid eye movements (REM) sleep (N2 and N3) and REM sleep. CAC was evaluated by means of linear spectral analysis, nonlinear symbolic analysis and complexity indexes. In both RES and CON groups, sympathetic modulation decreased and parasympathetic modulation increased during N2 and N3 compared to W and REM at D3, D5, D11, D14. Complexity analysis revealed a reduction in complexity during REM compared to NREM sleep in both DEP and CON After 8 days of moderate SD, cardiac autonomic dynamics, characterized by decreased sympathetic, and increased parasympathetic modulation, and higher cardiac complexity during NREM sleep, compared to W and REM, are preserved.

Characterizing dilute combustion instabilities in a multi-cylinder spark-ignited engine using symbolic analysis

Spark-ignited internal combustion engines have evolved considerably in recent years in response to increasingly stringent regulations for emissions and fuel economy. One new advanced engine strategy ustilizes high levels of exhaust gas recirculation (EGR) to reduce combustion temperatures, thereby increasing thermodynamic efficiency and reducing nitrogen oxide emissions. While this strategy can be highly effective, it also poses major control and design challenges due to the large combustion oscillations that develop at sufficiently high EGR levels. Previous research has documented that combustion instabilities can propagate between successive engine cycles in individual cylinders via self-generated feedback of reactive species and thermal energy in the retained residual exhaust gases. In this work, we use symbolic analysis to characterize multi-cylinder combustion oscillations in an experimental engine operating with external EGR. At low levels of EGR, intra-cylinder oscillations are clearly visible and appear to be associated with brief, intermittent coupling among cylinders. As EGR is increased further, a point is reached where all four cylinders lock almost completely in phase and alternate simultaneously between two distinct bi-stable combustion states. From a practical perspective, it is important to understand the causes of this phenomenon and develop diagnostics that might be applied to ameliorate its effects. We demonstrate here that two approaches for symbolizing the engine combustion measurements can provide useful probes for characterizing these instabilities.

Assessing levels of consciousness with symbolic analysis

'Covert consciousness' is a state in which consciousness is present without the capacity for behavioural response, and it can occur in patients with intraoperative awareness or unresponsive wakefulness syndrome. To detect and prevent this undesirable state, it is critical to develop a reliable neurobiological assessment of an individual's level of consciousness that is independent of behaviour. One such approach that shows potential is measuring surrogates of cortical communication in the brain using electroencephalography (EEG). EEG is practicable in clinical application, but involves many fundamental signal processing problems, including signal-to-noise ratio and high dimensional complexity. Symbolic analysis of EEG can mitigate these problems, improving the measurement of brain connectivity and the ability to successfully assess levels of consciousness. In this article, we review the problem of covert consciousness, basic neurobiological principles of consciousness, current methods of measuring brain connectivity and the advantages of symbolic processing, with a focus on symbolic transfer entropy (STE). Finally, we discuss recent advances and clinical applications of STE and other symbolic analyses to assess levels of consciousness.

Influence of type 2 diabetes on symbolic analysis and complexity of heart rate variability in men

BACKGROUND

Individuals with diabetes may develop cardiac autonomic dysfunction that may be evaluated by heart rate variability (HRV). The aim was evaluated heart rate variability (HRV) of individuals with type 2 diabetes, without cardiovascular autonomic neuropathy (CAN), in response to active postural maneuver by means of nonlinear analysis (symbolic analysis, Shannon and conditional entropy) and correlate HRV parameters between them, glycated hemoglobin and diabetes duration.

METHODS

Nineteen men with type 2 diabetes without CAN (T2D) and nineteen healthy men (CG), age-range from 40 to 60 years were studied. We assessed HRV in supine and orthostatic position using symbolic analysis (0V%, 1V%, 2LV% and 2UV%), Shannon and conditional entropy (SE and NCI).

RESULTS

In supine position T2D presented higher sympathetic modulation (0V%) than CG. However, there was not any difference between groups for indexes of complexity (SE and NCI). Furthermore, T2D presented a preserved response of cardiac autonomic modulation after active postural maneuver.

CONCLUSIONS

The present study showed that individuals with type 2 diabetes without CAN presented higher cardiac sympathetic modulation. However, the complexity of HRV was not influenced by imbalance of the autonomic modulation in individuals with type 2 diabetes. In addition, the response of autonomic nervous system in the heart remains preserved after active postural maneuver in individuals with type 2 diabetes, possibly due to the lack of CAN in this group.