Multiscale information storage of linear long-range correlated stochastic processes

Testing dynamic correlations and nonlinearity in bivariate time series through information measures and surrogate data analysis

The increasing availability of time series data depicting the evolution of physical system properties has prompted the development of methods focused on extracting insights into the system behavior over time, discerning whether it stems from deterministic or stochastic dynamical systems. Surrogate data testing plays a crucial role in this process by facilitating robust statistical assessments. This ensures that the observed results are not mere occurrences by chance, but genuinely reflect the inherent characteristics of the underlying system. The initial process involves formulating a null hypothesis, which is tested using surrogate data in cases where assumptions about the underlying distributions are absent. A discriminating statistic is then computed for both the original data and each surrogate data set. Significantly deviating values between the original data and the surrogate data ensemble lead to the rejection of the null hypothesis. In this work, we present various surrogate methods designed to assess specific statistical properties in random processes. Specifically, we introduce methods for evaluating the presence of autodependencies and nonlinear dynamics within individual processes, using Information Storage as a discriminating statistic. Additionally, methods are introduced for detecting coupling and nonlinearities in bivariate processes, employing the Mutual Information Rate for this purpose. The surrogate methods introduced are first tested through simulations involving univariate and bivariate processes exhibiting both linear and nonlinear dynamics. Then, they are applied to physiological time series of Heart Period (RR intervals) and respiratory flow (RESP) variability measured during spontaneous and paced breathing. Simulations demonstrated that the proposed methods effectively identify essential dynamical features of stochastic systems. The real data application showed that paced breathing, at low breathing rate, increases the predictability of the individual dynamics of RR and RESP and dampens nonlinearity in their coupled dynamics.

Respiratory modulation of the heart rate: A potential biomarker of cardiorespiratory function in human

In recent decade, wearable digital devices have shown potentials for the discovery of novel biomarkers of humans' physiology and behavior. Heart rate (HR) and respiration rate (RR) are most crucial bio-signals in humans' digital phenotyping research. HR is a continuous and non-invasive proxy to autonomic nervous system and ample evidence pinpoints the critical role of respiratory modulation of cardiac function. In the present study, we recorded longitudinal (7 days, 4.63 ± 1.52) HR and RR of 89 freely behaving human subjects (Female: 39, age 57.28 ± 5.67, Male: 50, age 58.48 ± 6.32) and analyzed their dynamics using linear models and information theoretic measures. While HR's linear and nonlinear characteristics were expressed within the plane of the HR-RR directed flow of information (HR→RR - RR→HR), their dynamics were determined by its RR→HR axis. More importantly, RR→HR quantified the effect of alcohol consumption on individuals' cardiorespiratory function independent of their consumed amount of alcohol, thereby signifying the presence of this habit in their daily life activities. The present findings provided evidence for the critical role of the respiratory modulation of HR, which was previously only studied in non-human animals. These results can contribute to humans' phenotyping research by presenting RR→HR as a digital diagnosis/prognosis marker of humans' cardiorespiratory pathology.

Time-varying information measures: an adaptive estimation of information storage with application to brain-heart interactions

Network Physiology is a rapidly growing field of study that aims to understand how physiological systems interact to maintain health. Within the information theory framework the information storage (IS) allows to measure the regularity and predictability of a dynamic process under stationarity assumption. However, this assumption does not allow to track over time the transient pathways occurring in the dynamical activity of a physiological system. To address this limitation, we propose a time-varying approach based on the recursive least squares algorithm (RLS) for estimating IS at each time instant, in non-stationary conditions. We tested this approach in simulated time-varying dynamics and in the analysis of electroencephalographic (EEG) signals recorded from healthy volunteers and timed with the heartbeat to investigate brain-heart interactions. In simulations, we show that the proposed approach allows to track both abrupt and slow changes in the information stored in a physiological system. These changes are reflected in its evolution and variability over time. The analysis of brain-heart interactions reveals marked differences across the cardiac cycle phases of the variability of the time-varying IS. On the other hand, the average IS values exhibit a weak modulation over parieto-occiptal areas of the scalp. Our study highlights the importance of developing more advanced methods for measuring IS that account for non-stationarity in physiological systems. The proposed time-varying approach based on RLS represents a useful tool for identifying spatio-temporal dynamics within the neurocardiac system and can contribute to the understanding of brain-heart interactions.

Comparison of discretization strategies for the model-free information-theoretic assessment of short-term physiological interactions

This work presents a comparison between different approaches for the model-free estimation of information-theoretic measures of the dynamic coupling between short realizations of random processes. The measures considered are the mutual information rate (MIR) between two random processes X and Y and the terms of its decomposition evidencing either the individual entropy rates of X and Y and their joint entropy rate, or the transfer entropies from X to Y and from Y to X and the instantaneous information shared by X and Y. All measures are estimated through discretization of the random variables forming the processes, performed either via uniform quantization (binning approach) or rank ordering (permutation approach). The binning and permutation approaches are compared on simulations of two coupled non-identical Hènon systems and on three datasets, including short realizations of cardiorespiratory (CR, heart period and respiration flow), cardiovascular (CV, heart period and systolic arterial pressure), and cerebrovascular (CB, mean arterial pressure and cerebral blood flow velocity) measured in different physiological conditions, i.e., spontaneous vs paced breathing or supine vs upright positions. Our results show that, with careful selection of the estimation parameters (i.e., the embedding dimension and the number of quantization levels for the binning approach), meaningful patterns of the MIR and of its components can be achieved in the analyzed systems. On physiological time series, we found that paced breathing at slow breathing rates induces less complex and more coupled CR dynamics, while postural stress leads to unbalancing of CV interactions with prevalent baroreflex coupling and to less complex pressure dynamics with preserved CB interactions. These results are better highlighted by the permutation approach, thanks to its more parsimonious representation of the discretized dynamic patterns, which allows one to explore interactions with longer memory while limiting the curse of dimensionality.

Information theoretic evidence for layer- and frequency-specific changes in cortical information processing under anesthesia

Nature relies on highly distributed computation for the processing of information in nervous systems across the entire animal kingdom. Such distributed computation can be more easily understood if decomposed into the three elementary components of information processing, i.e. storage, transfer and modification, and rigorous information theoretic measures for these components exist. However, the distributed computation is often also linked to neural dynamics exhibiting distinct rhythms. Thus, it would be beneficial to associate the above components of information processing with distinct rhythmic processes where possible. Here we focus on the storage of information in neural dynamics and introduce a novel spectrally-resolved measure of active information storage (AIS). Drawing on intracortical recordings of neural activity in ferrets under anesthesia before and after loss of consciousness (LOC) we show that anesthesia- related modulation of AIS is highly specific to different frequency bands and that these frequency-specific effects differ across cortical layers and brain regions. We found that in the high/low gamma band the effects of anesthesia result in AIS modulation only in the supergranular layers, while in the alpha/beta band the strongest decrease in AIS can be seen at infragranular layers. Finally, we show that the increase of spectral power at multiple frequencies, in particular at alpha and delta bands in frontal areas, that is often observed during LOC ('anteriorization') also impacts local information processing-but in a frequency specific way: Increases in isoflurane concentration induced a decrease in AIS in the alpha frequencies, while they increased AIS in the delta frequency range < 2Hz. Thus, the analysis of spectrally-resolved AIS provides valuable additional insights into changes in cortical information processing under anaesthesia.

Multiscale partial information decomposition of dynamic processes with short and long-range correlations: theory and application to cardiovascular control

OBJECTIVE

In this work, an analytical framework for the multiscale analysis of multivariate Gaussian processes is presented, whereby the computation of Partial Information Decomposition measures is achieved accounting for the simultaneous presence of short-term dynamics and long-range correlations.

APPROACH

We consider physiological time series mapping the activity of the cardiac, vascular and respiratory systems in the field of Network Physiology. In this context, the multiscale representation of transfer entropy within the network of interactions among Systolic arterial pressure (S), respiration (R) and heart period (H), as well as the decomposition into unique, redundant and synergistic contributions, is obtained using a Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This novel approach allows to quantify the directed information flow accounting for the simultaneous presence of short-term dynamics and long-range correlations among the analyzed processes. Additionally, it provides analytical expressions for the computation of the information measures, by exploiting the theory of state space models. The approach is first illustrated in simulated VARFI processes and then applied to H, S and R time series measured in healthy subjects monitored at rest and during mental and postural stress.

MAIN RESULTS

We demonstrate the ability of the VARFI modeling approach to account for the coexistence of short-term and long-range correlations in the study of multivariate processes. Physiologically, we show that postural stress induces larger redundant and synergistic effects from S and R to H at short time scales, while mental stress induces larger information transfer from S to H at longer time scales, thus evidencing the different nature of the two stressors.

SIGNIFICANCE

The proposed methodology allows to extract useful information about the dependence of the information transfer on the balance between short-term and long-range correlations in coupled dynamical systems, which cannot be observed using standard methods that do not consider long-range correlations.

Optimized Multiscale Entropy Model Based on Resting-State fMRI for Appraising Cognitive Performance in Healthy Elderly

Many studies have indicated that an entropy model can capture the dynamic characteristics of resting-state functional magnetic resonance imaging (rfMRI) signals. However, there are problems of subjectivity and lack of uniform standards in the selection of model parameters relying on experience when using the entropy model to analyze rfMRI. To address this issue, an optimized multiscale entropy (MSE) model was proposed to confirm the parameters objectively. All healthy elderly volunteers were divided into two groups, namely, excellent and poor, by the scores estimated through traditional scale tests before the rfMRI scan. The parameters of the MSE model were optimized with the help of sensitivity parameters such as receiver operating characteristic (ROC) and area under the ROC curve (AUC) in a comparison study between the two groups. The brain regions with significant differences in entropy values were considered biomarkers. Their entropy values were regarded as feature vectors to use as input for the probabilistic neural network in the classification of cognitive scores. Classification accuracy of 80.05% was obtained using machine learning. These results show that the optimized MSE model can accurately select the brain regions sensitive to cognitive performance and objectively select fixed parameters for MSE. This work was expected to provide the basis for entropy to test the cognitive scores of the healthy elderly.

Information Dynamics of Electric Field Intensity before and during the COVID-19 Pandemic

This work investigates the temporal statistical structure of time series of electric field (EF) intensity recorded with the aim of exploring the dynamical patterns associated with periods with different human activity in urban areas. The analyzed time series were obtained from a sensor of the EMF RATEL monitoring system installed in the campus area of the University of Novi Sad, Serbia. The sensor performs wideband cumulative EF intensity monitoring of all active commercial EF sources, thus including those linked to human utilization of wireless communication systems. Monitoring was performed continuously during the years 2019 and 2020, allowing us to investigate the effects on the patterns of EF intensity of varying conditions of human mobility, including regular teaching and exam activity within the campus, as well as limitations to mobility related to the COVID-19 pandemic. Time series analysis was performed using both simple statistics (mean and variance) and combining the information-theoretic measure of information storage (IS) with the method of surrogate data to quantify the regularity of EF dynamic patterns and detect the presence of nonlinear dynamics. Moreover, to assess the possible coexistence of dynamic behaviors across multiple temporal scales, IS analysis was performed over consecutive observation windows lasting one day, week, month, and year, respectively coarse grained at time scales of 6 min, 30 min, 2 h, and 1 day. Our results document that the EF intensity patterns of variability are modulated by the movement of people at daily, weekly, and monthly scales, and are blunted during periods of restricted mobility related to the COVID-19 pandemic. Mobility restrictions also affected significantly the regularity of the EF intensity time series, resulting in lower values of IS observed simultaneously with a loss of nonlinear dynamics. Thus, our analysis can be useful to investigate changes in the global patterns of human mobility both during pandemics or other types of events, and from this perspective may serve to implement strategies for safety assessment and for optimizing the design of networks of EF sensors.

Assessing Transfer Entropy in cardiovascular and respiratory time series under long-range correlations

Heart Period (H) results from the activity of several coexisting control mechanisms, involving Systolic Arterial Pressure (S) and Respiration (R), which operate across multiple time scales encompassing not only short-term dynamics but also long-range correlations. In this work, multiscale representation of Transfer Entropy (TE) and of its decomposition in the network of these three interacting processes is obtained by extending the multivariate approach based on linear parametric VAR models to the Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This approach allows to dissect the different contributions to cardiac dynamics accounting for the simultaneous presence of short and long term dynamics. The proposed method is first tested on simulations of a benchmark VARFI model and then applied to experimental data consisting of H, S and R time series measured in healthy subjects monitored at rest and during mental and postural stress. The results reveal that the proposed method can highlight the dependence of the information transfer on the balance between short-term and long-range correlations in coupled dynamical systems.

Feasibility of Linear Parametric Estimation of Dynamic Information Measures to assess Physiological Stress from Short-Term Cardiovascular Variability. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:290-293. [PMID: 34891293 DOI: 10.1109/embc46164.2021.9630697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Extensive efforts have been recently devoted to implement fast and reliable algorithms capable of assessing the physiological response of the organism to physiological stress. In this study, we propose the comparison between model-free and linear parametric methods as regards their ability to detect alterations in the dynamics and in the complexity of cardiovascular and respiratory variability evoked by postural and mental stress. Dynamic entropy (DE) and information storage (IS) measures were calculated on three physiological time-series, i.e. heart period, respiratory volume and systolic arterial pressure, on 61 healthy subjects monitored in resting conditions as well as during head-up tilt and while performing a mental arithmetic task. The results of the comparison suggest the feasibility of DE and IS measures computed from different physiological signals to discriminate among resting and stress states. If compared to the model-free algorithm, the faster linear method appears to be capable of detecting the same (or even more) statistically significant variations of DE or IS between resting and stress conditions, being thus in perspective more suitable for the integration within wearable devices. The computation of entropy indices extracted from multiple physiological signals acquired through wearables will allow a real-time stress assessment on people in daily-life situations.

Assessment of Cardiorespiratory Interactions during Apneic Events in Sleep via Fuzzy Kernel Measures of Information Dynamics

Apnea and other breathing-related disorders have been linked to the development of hypertension or impairments of the cardiovascular, cognitive or metabolic systems. The combined assessment of multiple physiological signals acquired during sleep is of fundamental importance for providing additional insights about breathing disorder events and the associated impairments. In this work, we apply information-theoretic measures to describe the joint dynamics of cardiorespiratory physiological processes in a large group of patients reporting repeated episodes of hypopneas, apneas (central, obstructive, mixed) and respiratory effort related arousals (RERAs). We analyze the heart period as the target process and the airflow amplitude as the driver, computing the predictive information, the information storage, the information transfer, the internal information and the cross information, using a fuzzy kernel entropy estimator. The analyses were performed comparing the information measures among segments during, immediately before and after the respiratory event and with control segments. Results highlight a general tendency to decrease of predictive information and information storage of heart period, as well as of cross information and information transfer from respiration to heart period, during the breathing disordered events. The information-theoretic measures also vary according to the breathing disorder, and significant changes of information transfer can be detected during RERAs, suggesting that the latter could represent a risk factor for developing cardiovascular diseases. These findings reflect the impact of different sleep breathing disorders on respiratory sinus arrhythmia, suggesting overall higher complexity of the cardiac dynamics and weaker cardiorespiratory interactions which may have physiological and clinical relevance.

Standard multiscale entropy reflects neural dynamics at mismatched temporal scales: What's signal irregularity got to do with it?

Multiscale Entropy (MSE) is used to characterize the temporal irregularity of neural time series patterns. Due to its' presumed sensitivity to non-linear signal characteristics, MSE is typically considered a complementary measure of brain dynamics to signal variance and spectral power. However, the divergence between these measures is often unclear in application. Furthermore, it is commonly assumed (yet sparingly verified) that entropy estimated at specific time scales reflects signal irregularity at those precise time scales of brain function. We argue that such assumptions are not tenable. Using simulated and empirical electroencephalogram (EEG) data from 47 younger and 52 older adults, we indicate strong and previously underappreciated associations between MSE and spectral power, and highlight how these links preclude traditional interpretations of MSE time scales. Specifically, we show that the typical definition of temporal patterns via "similarity bounds" biases coarse MSE scales-that are thought to reflect slow dynamics-by high-frequency dynamics. Moreover, we demonstrate that entropy at fine time scales-presumed to indicate fast dynamics-is highly sensitive to broadband spectral power, a measure dominated by low-frequency contributions. Jointly, these issues produce counterintuitive reflections of frequency-specific content on MSE time scales. We emphasize the resulting inferential problems in a conceptual replication of cross-sectional age differences at rest, in which scale-specific entropy age effects could be explained by spectral power differences at mismatched temporal scales. Furthermore, we demonstrate how such problems may be alleviated, resulting in the indication of scale-specific age differences in rhythmic irregularity. By controlling for narrowband contributions, we indicate that spontaneous alpha rhythms during eyes open rest transiently reduce broadband signal irregularity. Finally, we recommend best practices that may better permit a valid estimation and interpretation of neural signal irregularity at time scales of interest.

Multivariate and Multiscale Complexity of Long-Range Correlated Cardiovascular and Respiratory Variability Series

Assessing the dynamical complexity of biological time series represents an important topic with potential applications ranging from the characterization of physiological states and pathological conditions to the calculation of diagnostic parameters. In particular, cardiovascular time series exhibit a variability produced by different physiological control mechanisms coupled with each other, which take into account several variables and operate across multiple time scales that result in the coexistence of short term dynamics and long-range correlations. The most widely employed technique to evaluate the dynamical complexity of a time series at different time scales, the so-called multiscale entropy (MSE), has been proven to be unsuitable in the presence of short multivariate time series to be analyzed at long time scales. This work aims at overcoming these issues via the introduction of a new method for the assessment of the multiscale complexity of multivariate time series. The method first exploits vector autoregressive fractionally integrated (VARFI) models to yield a linear parametric representation of vector stochastic processes characterized by short- and long-range correlations. Then, it provides an analytical formulation, within the theory of state-space models, of how the VARFI parameters change when the processes are observed across multiple time scales, which is finally exploited to derive MSE measures relevant to the overall multivariate process or to one constituent scalar process. The proposed approach is applied on cardiovascular and respiratory time series to assess the complexity of the heart period, systolic arterial pressure and respiration variability measured in a group of healthy subjects during conditions of postural and mental stress. Our results document that the proposed methodology can detect physiologically meaningful multiscale patterns of complexity documented previously, but can also capture significant variations in complexity which cannot be observed using standard methods that do not take into account long-range correlations.

A Study of Brain Neuronal and Functional Complexities Estimated Using Multiscale Entropy in Healthy Young Adults

Brain complexity estimated using sample entropy and multiscale entropy (MSE) has recently gained much attention to compare brain function between diseased or neurologically impaired groups and healthy control groups. Using resting-state functional magnetic resonance imaging (rfMRI) blood oxygen-level dependent (BOLD) signals in a large cohort (n = 967) of healthy young adults, the present study maps neuronal and functional complexities estimated by using MSE of BOLD signals and BOLD phase coherence connectivity, respectively, at various levels of the brain’s organization. The functional complexity explores patterns in a higher dimension than neuronal complexity and may better discern changes in brain functioning. The leave-one-subject-out cross-validation method is used to predict fluid intelligence using neuronal and functional complexity MSE values as features. While a wide range of scales was selected with neuronal complexity, only the first three scales were selected with functional complexity. Fewer scales are advantageous as they preclude the need for long BOLD signals to calculate good estimates of MSE. The presented results corroborate with previous findings and provide a baseline for other studies exploring the use of MSE to examine changes in brain function related to aging, diseases, and clinical disorders.

Time, frequency and information domain analysis of short-term heart rate variability before and after focal and generalized seizures in epileptic children

OBJECTIVE

In this work we explore the potential of combining standard time and frequency domain indexes with novel information measures, to characterize pre- and post-ictal heart rate variability (HRV) in epileptic children, with the aim of differentiating focal and generalized epilepsy regarding the autonomic control mechanisms.

APPROACH

We analyze short-term HRV in 37 children suffering from generalized or focal epilepsy, monitored 10 s, 300 s, 600 s and 1800 s both before and after seizure episodes. Nine indexes are computed in time (mean, standard deviation of normal-to-normal intervals, root mean square of the successive differences (RMSSD)), frequency (low-to-high frequency power ratio LF/HF, normalized LF and HF power) and information (entropy, conditional entropy and self-entropy) domains. Focal and generalized epilepsy are compared through statistical analysis of the indexes and using linear discriminant analysis (LDA).

MAIN RESULTS

In children with focal epilepsy, early post-ictal phase is characterized by significant tachycardia, depressed HRV, increased LF power and LF/HF, and decreased complexity, progressively recovered across time windows after the episodes. Children with generalized seizures instead show significant tachycardia, lower RMSSD, higher LF power and LF/HF ratio before the seizure. These different behaviors are exploited by LDA analysis to separate focal and generalized epilepsy up to an accuracy of 75%. Results suggest a shift of the sympatho-vagal balance towards sympathetic dominance and vagal withdrawal, noticeable just after the termination of seizure episodes and then reverted in focal epilepsy, and persistent during inter-ictal and pre-ictal periods in generalized epilepsy.

SIGNIFICANCE

Our analysis helps in elucidating the pathophysiology of inter-ictal HRV autonomic control and the differential diagnosis of generalized and focal epilepsy. These findings may have clinical relevance since altered sympatho-vagal control can be related to a higher danger of morbidity and mortality, may reduce thresholds for life-threatening arrhythmias, and could be a biomarker of risk for sudden unexpected death in epilepsy.

A Parsimonious Granger Causality Formulation for Capturing Arbitrarily Long Multivariate Associations

High-frequency neuroelectric signals like electroencephalography (EEG) or magnetoencephalography (MEG) provide a unique opportunity to infer causal relationships between local activity of brain areas. While causal inference is commonly performed through classical Granger causality (GC) based on multivariate autoregressive models, this method may encounter important limitations (e.g., data paucity) in the case of high dimensional data from densely connected systems like the brain. Additionally, physiological signals often present long-range dependencies which commonly require high autoregressive model orders/number of parameters. We present a generalization of autoregressive models for GC estimation based on Wiener–Volterra decompositions with Laguerre polynomials as basis functions. In this basis, the introduction of only one additional global parameter allows to capture arbitrary long dependencies without increasing model order, hence retaining model simplicity, linearity and ease of parameters estimation. We validate our method in synthetic data generated from families of complex, densely connected networks and demonstrate superior performance as compared to classical GC. Additionally, we apply our framework to studying the directed human brain connectome through MEG data from 89 subjects drawn from the Human Connectome Project (HCP) database, showing that it is able to reproduce current knowledge as well as to uncover previously unknown directed influences between cortical and limbic brain regions.