Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes

Partial information decomposition for continuous variables based on shared exclusions: Analytical formulation and estimation

Describing statistical dependencies is foundational to empirical scientific research. For uncovering intricate and possibly nonlinear dependencies between a single target variable and several source variables within a system, a principled and versatile framework can be found in the theory of partial information decomposition (PID). Nevertheless, the majority of existing PID measures are restricted to categorical variables, while many systems of interest in science are continuous. In this paper, we present a novel analytic formulation for continuous redundancy-a generalization of mutual information-drawing inspiration from the concept of shared exclusions in probability space as in the discrete PID definition of I_{∩}^{sx}. Furthermore, we introduce a nearest-neighbor-based estimator for continuous PID and showcase its effectiveness by applying it to a simulated energy management system provided by the Honda Research Institute Europe GmbH. This work bridges the gap between the measure-theoretically postulated existence proofs for a continuous I_{∩}^{sx} and its practical application to real-world scientific problems.

Sampling bias corrections for accurate neural measures of redundant, unique, and synergistic information

Shannon Information theory has long been a tool of choice to measure empirically how populations of neurons in the brain encode information about cognitive variables. Recently, Partial Information Decomposition (PID) has emerged as principled way to break down this information into components identifying not only the unique information carried by each neuron, but also whether relationships between neurons generate synergistic or redundant information. While it has been long recognized that Shannon information measures on neural activity suffer from a (mostly upward) limited sampling estimation bias, this issue has largely been ignored in the burgeoning field of PID analysis of neural activity. We used simulations to investigate the limited sampling bias of PID computed from discrete probabilities (suited to describe neural spiking activity). We found that PID suffers from a large bias that is uneven across components, with synergy by far the most biased. Using approximate analytical expansions, we found that the bias of synergy increases quadratically with the number of discrete responses of each neuron, whereas the bias of unique and redundant information increase only linearly or sub-linearly. Based on the understanding of the PID bias properties, we developed simple yet effective procedures that correct for the bias effectively, and that improve greatly the PID estimation with respect to current state-of-the-art procedures. We apply these PID bias correction procedures to datasets of 53117 pairs neurons in auditory cortex, posterior parietal cortex and hippocampus of mice performing cognitive tasks, deriving precise estimates and bounds of how synergy and redundancy vary across these brain regions.

A Stepwise Multivariate Granger Causality Method for Constructing Hierarchical Directed Brain Functional Network

The directed brain functional network construction gives us the new insights into the relationships between brain regions from the causality point of view. The Granger causality analysis is one of the powerful methods to model the directed network. The complex brain network is also hierarchically constructed, which is particularly suited to facilitate segregated functions and the global integration of the segregated functions. Therefore, it is of great interest to explore new approach to model the hierarchical architecture of the directed network. In the present study, we proposed a new approach, namely, stepwise multivariate Granger causality (SMGC), considering both the directed and hierarchical features of brain functional network to explore the stepwise causal relationship in the network. The simulation study demonstrated that the diverse and complex hierarchical organization could be embedded in the apparently simple directed network. The proposed SMGC method could capture the multiple hierarchy of the directed network. When applying to the real functional magnetic resonance imaging (fMRI) datasets, the core triple resting-state networks in human brain showed within-network directed connections in the first-level directed network and rich and diverse between-network pathways in the second-level hierarchical network. The default mode network (DMN) had a prominent role in the resting-state acting as both the causal source and the important relay station. Further exploratory research on the adaption of directed hierarchical network in athletes suggested the enhanced bidirectional communication between the DMN and the central executive network (CEN) and the enhanced directed connections from the salience network (SN) to the CEN in the athlete group. The SMGC approach is capable of capturing the hierarchical architecture of the brain directed functional network, which refreshes the new stepwise causal relationship in the directed network. This might shed light on the potential application for exploring the altered hierarchical organization of brain directed network in neuropsychiatric disorders.

Confounding Factor Analysis for Vocal Fold Oscillations

This paper provides a methodology to better understand the relationships between different aspects of vocal fold motion, which are used as features in machine learning-based approaches for detecting respiratory infections from voice recordings. The relationships are derived through a joint multivariate analysis of the vocal fold oscillations of speakers. Specifically, the multivariate setting explores the displacements and velocities of the left and right vocal folds derived from recordings of five extended vowel sounds for each speaker (/aa/, /iy/, /ey/, /uw/, and /ow/). In this multivariate setting, the differences between the bivariate and conditional interactions are analyzed by information-theoretic quantities based on transfer entropy. Incorporation of the conditional quantities reveals information regarding the confounding factors that can influence the statistical interactions among other pairs of variables. This is demonstrated on a vector autoregressive process where the analytical derivations can be carried out. As a proof of concept, the methodology is applied on a clinically curated dataset of COVID-19. The findings suggest that the interaction between the vocal fold oscillations can change according to individuals and presence of any respiratory infection, such as COVID-19. The results are important in the sense that the proposed approach can be utilized to determine the selection of appropriate features as a supplementary or early detection tool in voice-based diagnostics in future studies.

Investigating Dynamic High-Order Interactions in Physiological Networks through Predictive Information Decomposition

We present an approach to assess redundant and synergistic interactions in network systems via the information-theoretic analysis of multivariate physiological processes. The approach sets up a strategy to decompose the information shared between the present states of a group of random processes and their own past states into unique contributions arising from the past of subgroups of processes and redundant and synergistic contributions arising from the dynamic interaction among the subgroups. The method is illustrated in a theoretical example of linearly interacting Gaussian processes, showing that redundancy and synergy are related mostly to unidirectional coupling and to bidirectional coupling with internal dynamics. It is then applied to the network of short-term heart period, arterial pressure and respiratory variability probed in healthy subjects, showing that redundancy and synergy prevail respectively in cardiorespiratory interactions and in cardiovascular interactions in the resting state, and that postural stress increases the predictive information and the redundancy of physiological interactions.

Connectivity Analysis in EEG Data: A Tutorial Review of the State of the Art and Emerging Trends

Understanding how different areas of the human brain communicate with each other is a crucial issue in neuroscience. The concepts of structural, functional and effective connectivity have been widely exploited to describe the human connectome, consisting of brain networks, their structural connections and functional interactions. Despite high-spatial-resolution imaging techniques such as functional magnetic resonance imaging (fMRI) being widely used to map this complex network of multiple interactions, electroencephalographic (EEG) recordings claim high temporal resolution and are thus perfectly suitable to describe either spatially distributed and temporally dynamic patterns of neural activation and connectivity. In this work, we provide a technical account and a categorization of the most-used data-driven approaches to assess brain-functional connectivity, intended as the study of the statistical dependencies between the recorded EEG signals. Different pairwise and multivariate, as well as directed and non-directed connectivity metrics are discussed with a pros-cons approach, in the time, frequency, and information-theoretic domains. The establishment of conceptual and mathematical relationships between metrics from these three frameworks, and the discussion of novel methodological approaches, will allow the reader to go deep into the problem of inferring functional connectivity in complex networks. Furthermore, emerging trends for the description of extended forms of connectivity (e.g., high-order interactions) are also discussed, along with graph-theory tools exploring the topological properties of the network of connections provided by the proposed metrics. Applications to EEG data are reviewed. In addition, the importance of source localization, and the impacts of signal acquisition and pre-processing techniques (e.g., filtering, source localization, and artifact rejection) on the connectivity estimates are recognized and discussed. By going through this review, the reader could delve deeply into the entire process of EEG pre-processing and analysis for the study of brain functional connectivity and learning, thereby exploiting novel methodologies and approaches to the problem of inferring connectivity within complex networks.

White-matter degradation and dynamical compensation support age-related functional alterations in human brain

Structural connectivity of the brain at different ages is analyzed using diffusion-weighted magnetic resonance imaging (MRI) data. The largest decrease of streamlines is found in frontal regions and for long inter-hemispheric links. The average length of the tracts also decreases, but the clustering is unaffected. From functional MRI we identify age-related changes of dynamic functional connectivity (dFC) and spatial covariation features of functional connectivity (FC) links captured by metaconnectivity. They indicate more stable dFC, but wider range and variance of MC, whereas static features of FC did not show any significant differences with age. We implement individual connectivity in whole-brain models and test several hypotheses for the mechanisms of operation among underlying neural system. We demonstrate that age-related functional fingerprints are only supported if the model accounts for: (i) compensation of the individual brains for the overall loss of structural connectivity and (ii) decrease of propagation velocity due to the loss of myelination. We also show that with these 2 conditions, it is sufficient to decompose the time-delays as bimodal distribution that only distinguishes between intra- and inter-hemispheric delays, and that the same working point also captures the static FC the best, and produces the largest variability at slow time-scales.

Multi-Scale Characteristics of Investor Sentiment Transmission Based on Wavelet, Transfer Entropy and Network Analysis

Investor sentiment transmission is significantly influential over financial markets. Prior studies do not reach a consensus about the multi-scale transmission patterns of investor sentiment. Our study proposed a composite set of methods based on wavelet, transfer entropy, and network analysis to explore the transmission patterns of investor sentiment among firms. By taking 137 new energy vehicle-related listed firms as an example, the results show three key findings: (1) the transmission of investor sentiment presents more active in the short term and takes place in a local range; (2) the transmission of investor sentiment presents patterns of continuity and growth from short term to long term; and (3) the transmission patterns of investor sentiment will have specific evolutions from short term to long term. Suggestions are offered to investors, managers and policymakers to better monitor the financial market using investor sentiment transmission.

Multiscale partial information decomposition of dynamic processes with short and long-range correlations: theory and application to cardiovascular control

OBJECTIVE

In this work, an analytical framework for the multiscale analysis of multivariate Gaussian processes is presented, whereby the computation of Partial Information Decomposition measures is achieved accounting for the simultaneous presence of short-term dynamics and long-range correlations.

APPROACH

We consider physiological time series mapping the activity of the cardiac, vascular and respiratory systems in the field of Network Physiology. In this context, the multiscale representation of transfer entropy within the network of interactions among Systolic arterial pressure (S), respiration (R) and heart period (H), as well as the decomposition into unique, redundant and synergistic contributions, is obtained using a Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This novel approach allows to quantify the directed information flow accounting for the simultaneous presence of short-term dynamics and long-range correlations among the analyzed processes. Additionally, it provides analytical expressions for the computation of the information measures, by exploiting the theory of state space models. The approach is first illustrated in simulated VARFI processes and then applied to H, S and R time series measured in healthy subjects monitored at rest and during mental and postural stress.

MAIN RESULTS

We demonstrate the ability of the VARFI modeling approach to account for the coexistence of short-term and long-range correlations in the study of multivariate processes. Physiologically, we show that postural stress induces larger redundant and synergistic effects from S and R to H at short time scales, while mental stress induces larger information transfer from S to H at longer time scales, thus evidencing the different nature of the two stressors.

SIGNIFICANCE

The proposed methodology allows to extract useful information about the dependence of the information transfer on the balance between short-term and long-range correlations in coupled dynamical systems, which cannot be observed using standard methods that do not consider long-range correlations.

Pairwise and higher-order measures of brain-heart interactions in children with temporal lobe epilepsy

OBJECTIVE

While it is well-known that epilepsy has a clear impact on the activity of both the central nervous system (CNS) and the autonomic nervous system (ANS), its role on the complex interplay between CNS and ANS has not been fully elucidated yet. In this work, pairwise and higher-order predictability measures based on the concepts of Granger causality (GC) and Partial Information Decomposition (PID) were applied on time series of electroencephalographic (EEG) brain wave amplitude and heart rate variability (HRV) in order to investigate directed brain-heart interactions associated with the occurrence of focal epilepsy.

APPROACH

HRV and the envelopes of δ and α EEG activity recorded from ipsilateral (ipsi-EEG) and contralateral (contra-EEG) scalp regions were analyzed in 18 children suffering from temporal lobe epilepsy monitored during pre-ictal, ictal and post-ictal periods. After linear parametric model identification, we compared pairwise GC measures computed between HRV and a single EEG component with PID measures quantifying the unique, redundant and synergistic information transferred from ipsi-EEG and contra-EEG to HRV.

MAIN RESULTS

The analysis of GC revealed a dominance of the information transfer from EEG to HRV and negligible transfer from HRV to EEG, suggesting that CNS activities drive the ANS modulation of the heart rhythm, but did not evidence clear differences between δ and α rhythms, ipsi-EEG and contra-EEG, or pre- and post-ictal periods. On the contrary, PID revealed that epileptic seizures induce a reorganization of the interactions from brain to heart, as the unique predictability of HRV originated from the ipsi-EEG for the δ waves and from the contra-EEG for the α waves in the pre-ictal phase, while these patterns were reversed after the seizure.

SIGNIFICANCE

These results highlight the importance of considering higher-order interactions elicited by PID for the study of the neuro-autonomic effects of focal epilepsy, and may have neurophysiological and clinical implications.

Multiscale Information Decomposition of Long Memory Processes: Application to Plateau Waves of Intracranial Pressure

Traumatic Brain Injury (TBI) patients present high levels of physical stress, which in some situations can manifest as Plateau Wave (PW) episodes. This intense stress phenomenon can be evidenced by Heart Rate Variability (HRV). Thus, the multivariate and simultaneous analysis of cardio-cerebrovascular oscillations, involving the RR intervals, mean arterial pressure (MAP) and the amplitude of intracranial pressure (AMP), will be useful to understand the interconnections between body signals, allowing the interpretation of the combined activity of pathophysiological mechanisms. In this work, the multiscale representation of the Transfer Entropy (TE) and of its decomposition in the network of these three interacting processes is obtained, based on a Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This method allows to assess directed interactions and to quantify the information flow accounting for the simultaneous presence of short-term dynamics and long-range correlations. The results show that the baseline RR, but not MAP can provide information about the possibility of a PW arising. During PW, the long-term correlations highlight synergistic interactions between MAP and AMP processes on RR. The multiscale decomposition of the information along with the incorporation of the long term correlations allowed a better description of HRV during PW, highlighting the fact that the HRV mirrors this cerebrovascular phenomena.

Phase irregularity: A conceptually simple and efficient approach to characterize electroencephalographic recordings from epilepsy patients

The severe neurological disorder epilepsy affects almost 1% of the world population. For patients who suffer from pharmacoresistant focal-onset epilepsy, electroencephalographic (EEG) recordings are essential for the localization of the brain area where seizures start. Apart from the visual inspection of the recordings, quantitative EEG signal analysis techniques proved to be useful for this purpose. Among other features, regularity versus irregularity and phase coherence versus phase independence allowed characterizing brain dynamics from the measured EEG signals. Can phase irregularities also characterize brain dynamics? To address this question, we use the univariate coefficient of phase velocity variation, defined as the ratio of phase velocity standard deviation and the mean phase velocity. Beyond that, as a bivariate measure we use the classical mean phase coherence to quantify the degree of phase locking. All phase-based measures are combined with surrogates to test null hypotheses about the dynamics underlying the signals. In the first part of our analysis, we use the Rössler model system to study our approach under controlled conditions. In the second part, we use the Bern-Barcelona EEG database which consists of focal and nonfocal signals extracted from seizure-free recordings. Focal signals are recorded from brain areas where the first seizure EEG signal changes can be detected, and nonfocal signals are recorded from areas that are not involved in the seizure at its onset. Our results show that focal signals have less phase variability and more phase coherence than nonfocal signals. Once combined with surrogates, the mean phase velocity proved to have the highest discriminative power between focal and nonfocal signals. In conclusion, conceptually simple and easy to compute phase-based measures can help to detect features induced by epilepsy from EEG signals. This holds not only for the classical mean phase coherence but even more so for univariate measures of phase irregularity.

Information decomposition in the frequency domain: a new framework to study cardiovascular and cardiorespiratory oscillations

While cross-spectral and information-theoretic approaches are widely used for the multivariate analysis of physiological time series, their combined utilization is far less developed in the literature. This study introduces a framework for the spectral decomposition of multivariate information measures, which provides frequency-specific quantifications of the information shared between a target and two source time series and of its expansion into amounts related to how the sources contribute to the target dynamics with unique, redundant and synergistic information. The framework is illustrated in simulations of linearly interacting stochastic processes, showing how it allows us to retrieve amounts of information shared by the processes within specific frequency bands which are otherwise not detectable by time-domain information measures, as well as coupling features which are not detectable by spectral measures. Then, it is applied to the time series of heart period, systolic and diastolic arterial pressure and respiration variability measured in healthy subjects monitored in the resting supine position and during head-up tilt. We show that the spectral measures of unique, redundant and synergistic information shared by these variability series, integrated within specific frequency bands of physiological interest and reflect the mechanisms of short-term regulation of cardiovascular and cardiorespiratory oscillations and their alterations induced by the postural stress. This article is part of the theme issue 'Advanced computation in cardiovascular physiology: new challenges and opportunities'.

Assessing Transfer Entropy in cardiovascular and respiratory time series under long-range correlations

Heart Period (H) results from the activity of several coexisting control mechanisms, involving Systolic Arterial Pressure (S) and Respiration (R), which operate across multiple time scales encompassing not only short-term dynamics but also long-range correlations. In this work, multiscale representation of Transfer Entropy (TE) and of its decomposition in the network of these three interacting processes is obtained by extending the multivariate approach based on linear parametric VAR models to the Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This approach allows to dissect the different contributions to cardiac dynamics accounting for the simultaneous presence of short and long term dynamics. The proposed method is first tested on simulations of a benchmark VARFI model and then applied to experimental data consisting of H, S and R time series measured in healthy subjects monitored at rest and during mental and postural stress. The results reveal that the proposed method can highlight the dependence of the information transfer on the balance between short-term and long-range correlations in coupled dynamical systems.

A new Framework for the Spectral Information Decomposition of Multivariate Gaussian Processes

Different information-theoretic measures are available in the literature for the study of pairwise and higher-order interactions in multivariate dynamical systems. While these measures operate in the time domain, several physiological and non-physiological systems exhibit a rich oscillatory content that is typically analyzed in the frequency domain through spectral and cross-spectral approaches. For Gaussian systems, the relation between information and spectral measures has been established considering coupling and causality measures, but not for higher-order interactions. To fill this gap, in this work we introduce an information-theoretic framework in the frequency domain to quantify the information shared between a target process and two sources, even multivariate, and to highlight the presence of redundancy and synergy in the analyzed dynamical system. Firstly, we simulate different linear interacting processes by showing the capability of the proposed framework to retrieve amounts of information shared by the processes in specific frequency bands which are not detectable by the related time-domain measures. Then, the framework is applied on EEG time series representative of the brain activity during a motor execution task in a group of healthy subjects.

Redundant Information Neural Estimation

We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the “redundant information”. We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, we show that optimizing over functions enables the approximation of the redundant information for high-dimensional and continuous predictors. We demonstrate this on high-dimensional image classification and motor-neuroscience tasks.

Information Flow Between Heart Rhythm, Repolarization, and the Diastolic Interval Series for Healthy Individuals and LQTS1 Patients

Using information theoretic measures, relations between heart rhythm, repolarization in the tissue of the heart, and the diastolic interval time series are analyzed. These processes are a fragment of the cardiovascular physiological network. A comparison is made between the results for 84 (42 women) healthy individuals and 65 (45 women) long QT syndrome type 1 (LQTS1) patients. Self-entropy, transfer entropy, and joint transfer entropy are calculated for the three time series and their combinations. The results for self-entropy indicate the well-known result that regularity of heart rhythm for healthy individuals is larger than that of QT interval series. The flow of information depends on the direction with the flow from the heart rhythm to QT dominating. In LQTS1 patients, however, our results indicate that information flow in the opposite direction may occur-a new result. The information flow from the heart rhythm to QT dominates, which verifies the asymmetry seen by Porta et al. in the variable tilt angle experiment. The amount of new information and self-entropy for LQTS1 patients is smaller than that for healthy individuals. However, information transfers from RR to QT and from DI to QT are larger in the case of LQTS1 patients.

Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators

One of the most challenging problems in the study of complex dynamical systems is to find the statistical interdependencies among the system components. Granger causality (GC) represents one of the most employed approaches, based on modeling the system dynamics with a linear vector autoregressive (VAR) model and on evaluating the information flow between two processes in terms of prediction error variances. In its most advanced setting, GC analysis is performed through a state-space (SS) representation of the VAR model that allows to compute both conditional and unconditional forms of GC by solving only one regression problem. While this problem is typically solved through Ordinary Least Square (OLS) estimation, a viable alternative is to use Artificial Neural Networks (ANNs) implemented in a simple structure with one input and one output layer and trained in a way such that the weights matrix corresponds to the matrix of VAR parameters. In this work, we introduce an ANN combined with SS models for the computation of GC. The ANN is trained through the Stochastic Gradient Descent L1 (SGD-L1) algorithm, and a cumulative penalty inspired from penalized regression is applied to the network weights to encourage sparsity. Simulating networks of coupled Gaussian systems, we show how the combination of ANNs and SGD-L1 allows to mitigate the strong reduction in accuracy of OLS identification in settings of low ratio between number of time series points and of VAR parameters. We also report how the performances in GC estimation are influenced by the number of iterations of gradient descent and by the learning rate used for training the ANN. We recommend using some specific combinations for these parameters to optimize the performance of GC estimation. Then, the performances of ANN and OLS are compared in terms of GC magnitude and statistical significance to highlight the potential of the new approach to reconstruct causal coupling strength and network topology even in challenging conditions of data paucity. The results highlight the importance of of a proper selection of regularization parameter which determines the degree of sparsity in the estimated network. Furthermore, we apply the two approaches to real data scenarios, to study the physiological network of brain and peripheral interactions in humans under different conditions of rest and mental stress, and the effects of the newly emerged concept of remote synchronization on the information exchanged in a ring of electronic oscillators. The results highlight how ANNs provide a mesoscopic description of the information exchanged in networks of multiple interacting physiological systems, preserving the most active causal interactions between cardiovascular, respiratory and brain systems. Moreover, ANNs can reconstruct the flow of directed information in a ring of oscillators whose statistical properties can be related to those of physiological networks.

Multiscale multivariate transfer entropy and application to functional corticocortical coupling

Objective. Complex biological systems consist of multi-level mechanism in terms of within- and cross-subsystems correlations, and they are primarily manifested in terms of connectivity, multiscale properties, and nonlinearity. Existing studies have each only explored one aspect of the functional corticocortical coupling (FCCC), which has some limitations in portraying the complexity of multivariable systems. The present study investigated the direct interactions of brain networks at multiple time scales.Approach. We extended the multivariate transfer entropy (MuTE) method and proposed a novel method, named multiscale multivariate transfer entropy (MSMVTE), to explore the direct interactions of brain networks across multiple time scale. To verify this aim, we introduced three simulation models and compared them with multiscale transfer entropy (MSTE) and MuTE methods. We then applied MSMVTE method to analyze FCCC during a unilateral right-hand steady-state force task.Main results. Simulation results showed that the MSMVTE method, compared with MSTE and MuTE methods, better detected direct interactions and avoid the spurious effects of indirect relationships. Further analysis of experimental data showed that the connectivity from left premotor/sensorimotor cortex to right premotor/sensorimotor cortex was significantly higher than that of opposite directionality. Furthermore, the connectivities from central motor areas to both sides of premotor/sensorimotor areas were higher than those of opposite directionalities. Additionally, the maximum coupling strength was found to occur at a specific scale (3-10).Significance. Simulation results confirmed the effectiveness of the MSMVTE method to describe direct relationships and multiscale characteristics in complex systems. The enhancement of FCCC reflects the interaction of more extended activation in cortical motor regions. Additionally, the neurodynamic process of brain depends not only on emergent behavior at small scales, but also on the constraining effects of the activity at large scales. Taken together, our findings provide a basis for better understanding dynamics in brain networks.

Diagnosing Schizophrenia Using Effective Connectivity of Resting-State EEG Data

Schizophrenia is a serious mental illness associated with neurobiological deficits. Even though the brain activities during tasks (i.e., P300 activities) are considered as biomarkers to diagnose schizophrenia, brain activities at rest have the potential to show an inherent dysfunctionality in schizophrenia and can be used to understand the cognitive deficits in these patients. In this study, we developed a machine learning algorithm (MLA) based on eyes closed resting-state electroencephalogram (EEG) datasets, which record the neural activity in the absence of any tasks or external stimuli given to the subjects, aiming to distinguish schizophrenic patients (SCZs) from healthy controls (HCs). The MLA has two steps. In the first step, symbolic transfer entropy (STE), which is a measure of effective connectivity, is applied to resting-state EEG data. In the second step, the MLA uses the STE matrix to find a set of features that can successfully discriminate SCZ from HC. From the results, we found that the MLA could achieve a total accuracy of 96.92%, with a sensitivity of 95%, a specificity of 98.57%, precision of 98.33%, F1-score of 0.97, and Matthews correlation coefficient (MCC) of 0.94 using only 10 out of 1900 STE features, which implies that the STE matrix extracted from resting-state EEG data may be a promising tool for the clinical diagnosis of schizophrenia.

Local Granger causality

Granger causality (GC) is a statistical notion of causal influence based on prediction via linear vector autoregression. For Gaussian variables it is equivalent to transfer entropy, an information-theoretic measure of time-directed information transfer between jointly dependent processes. We exploit such equivalence and calculate exactly the local Granger causality, i.e., the profile of the information transferred from the driver to the target process at each discrete time point; in this frame, GC is the average of its local version. We show that the variability of the local GC around its mean relates to the interplay between driver and innovation (autoregressive noise) processes, and it may reveal transient instances of information transfer not detectable from its average values. Our approach offers a robust and computationally fast method to follow the information transfer along the time history of linear stochastic processes, as well as of nonlinear complex systems studied in the Gaussian approximation.

Quantifying Dynamical High-Order Interdependencies From the O-Information: An Application to Neural Spiking Dynamics

We address the problem of efficiently and informatively quantifying how multiplets of variables carry information about the future of the dynamical system they belong to. In particular we want to identify groups of variables carrying redundant or synergistic information, and track how the size and the composition of these multiplets changes as the collective behavior of the system evolves. In order to afford a parsimonious expansion of shared information, and at the same time control for lagged interactions and common effect, we develop a dynamical, conditioned version of the O-information, a framework recently proposed to quantify high-order interdependencies via multivariate extension of the mutual information. The dynamic O-information, here introduced, allows to separate multiplets of variables which influence synergistically the future of the system from redundant multiplets. We apply this framework to a dataset of spiking neurons from a monkey performing a perceptual discrimination task. The method identifies synergistic multiplets that include neurons previously categorized as containing little relevant information individually.

Transfer entropies within dynamical effects framework

Transfer entropy (TE) is widely used in time-series analysis to detect causal couplings between temporally evolving objects. As a coupling strength quantifier, the TE alone often seems insufficient, raising the question of its further interpretations. Here the TE is related to dynamical causal effects (DCEs) which quantify long-term responses of a coupling recipient to variations in a coupling source or in a coupling itself: Detailed relationships are established for a paradigmatic stochastic dynamical system of bidirectionally coupled linear overdamped oscillators, their practical applications and possible extensions are discussed. It is shown that two widely used versions of the TE (original and infinite-history) can become qualitatively distinct, diverging to different long-term DCEs.

Information Dynamics Analysis: A new approach based on Sparse Identification of Linear Parametric Models

The framework of information dynamics allows to quantify different aspects of the statistical structure of multivariate processes reflecting the temporal dynamics of a complex network. The information transfer from one process to another can be quantified through Transfer Entropy, and under the assumption of joint Gaussian variables it is strictly related to the concept of Granger Causality (GC). According to the most recent developments in the field, the computation of GC entails representing the processes through a Vector Autoregressive (VAR) model and a state space (SS) model typically identified by means of the Ordinary Least Squares (OLS). In this work, we propose a new identification approach for the VAR and SS models, based on Least Absolute Shrinkage and Selection Operator (LASSO), that has the advantages of maintaining good accuracy even when few data samples are available and yielding as output a sparse matrix of estimated information transfer. The performances of LASSO identification were first tested and compared to those of OLS by a simulation study and then validated on real electroencephalographic (EEG) signals recorded during a motor imagery task. Both studies indicated that LASSO, under conditions of data paucity, provides better performances in terms of network structure. Given the general nature of the model, this work opens the way to the use of LASSO regression for the computation of several measures of information dynamics currently in use in computational neuroscience.

Estimating Conditional Transfer Entropy in Time Series Using Mutual Information and Nonlinear Prediction

We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of new information and improvement of the prediction accuracy provided by the variables. Then, using a greedy approach, the most informative subsets are selected in an iterative way. The algorithm terminates, when the highest ranked variable is not able to significantly improve the accuracy of the prediction as compared to that obtained using the existing selected subsets. In a simulation study, we compare our estimator to existing state-of-the-art methods at different data lengths and directed dependencies strengths. It is demonstrated that the proposed estimator has a significantly higher accuracy than that of existing methods, especially for the difficult case, where the data are highly correlated and coupled. Moreover, we show its false detection of directed dependencies due to instantaneous couplings effect is lower than that of existing measures. We also show applicability of the proposed estimator on real intracranial electroencephalography data.

Synergistic Information Transfer in the Global System of Financial Markets

Uncovering dynamic information flow between stock market indices has been the topic of several studies which exploited the notion of transfer entropy or Granger causality, its linear version. The output of the transfer entropy approach is a directed weighted graph measuring the information about the future state of each target provided by the knowledge of the state of each driving stock market index. In order to go beyond the pairwise description of the information flow, thus looking at higher order informational circuits, here we apply the partial information decomposition to triplets consisting of a pair of driving markets (belonging to America or Europe) and a target market in Asia. Our analysis, on daily data recorded during the years 2000 to 2019, allows the identification of the synergistic information that a pair of drivers carry about the target. By studying the influence of the closing returns of drivers on the subsequent overnight changes of target indexes, we find that (i) Korea, Tokyo, Hong Kong, and Singapore are, in order, the most influenced Asian markets; (ii) US indices SP500 and Russell are the strongest drivers with respect to the bivariate Granger causality; and (iii) concerning higher order effects, pairs of European and American stock market indices play a major role as the most synergetic three-variables circuits. Our results show that the Synergy, a proxy of higher order predictive information flow rooted in information theory, provides details that are complementary to those obtained from bivariate and global Granger causality, and can thus be used to get a better characterization of the global financial system.

Information Transfer in Linear Multivariate Processes Assessed through Penalized Regression Techniques: Validation and Application to Physiological Networks

The framework of information dynamics allows the dissection of the information processed in a network of multiple interacting dynamical systems into meaningful elements of computation that quantify the information generated in a target system, stored in it, transferred to it from one or more source systems, and modified in a synergistic or redundant way. The concepts of information transfer and modification have been recently formulated in the context of linear parametric modeling of vector stochastic processes, linking them to the notion of Granger causality and providing efficient tools for their computation based on the state-space (SS) representation of vector autoregressive (VAR) models. Despite their high computational reliability these tools still suffer from estimation problems which emerge, in the case of low ratio between data points available and the number of time series, when VAR identification is performed via the standard ordinary least squares (OLS). In this work we propose to replace the OLS with penalized regression performed through the Least Absolute Shrinkage and Selection Operator (LASSO), prior to computation of the measures of information transfer and information modification. First, simulating networks of several coupled Gaussian systems with complex interactions, we show that the LASSO regression allows, also in conditions of data paucity, to accurately reconstruct both the underlying network topology and the expected patterns of information transfer. Then we apply the proposed VAR-SS-LASSO approach to a challenging application context, i.e., the study of the physiological network of brain and peripheral interactions probed in humans under different conditions of rest and mental stress. Our results, which document the possibility to extract physiologically plausible patterns of interaction between the cardiovascular, respiratory and brain wave amplitudes, open the way to the use of our new analysis tools to explore the emerging field of Network Physiology in several practical applications.

Disambiguating the role of blood flow and global signal with partial information decomposition

Global signal (GS) is an ubiquitous construct in resting state functional magnetic resonance imaging (rs-fMRI), associated to nuisance, but containing by definition most of the neuronal signal. Global signal regression (GSR) effectively removes the impact of physiological noise and other artifacts, but at the same time it alters correlational patterns in unpredicted ways. Performing GSR taking into account the underlying physiology (mainly the blood arrival time) has been proven to be beneficial. From these observations we aimed to: 1) characterize the effect of GSR on network-level functional connectivity in a large dataset; 2) assess the complementary role of global signal and vessels; and 3) use the framework of partial information decomposition to further look into the joint dynamics of the global signal and vessels, and their respective influence on the dynamics of cortical areas. We observe that GSR affects intrinsic connectivity networks in the connectome in a non-uniform way. Furthermore, by estimating the predictive information of blood flow and the global signal using partial information decomposition, we observe that both signals are present in different amounts across intrinsic connectivity networks. Simulations showed that differences in blood arrival time can largely explain this phenomenon, while using hemodynamic and calcium mouse recordings we were able to confirm the presence of vascular effects, as calcium recordings lack hemodynamic information. With these results we confirm network-specific effects of GSR and the importance of taking blood flow into account for improving de-noising methods. Additionally, and beyond the mere issue of data denoising, we quantify the diverse and complementary effect of global and vessel BOLD signals on the dynamics of cortical areas.

Respiratory Sinus Arrhythmia Mechanisms in Young Obese Subjects

Autonomic nervous system (ANS) activity and imbalance between its sympathetic and parasympathetic components are important factors contributing to the initiation and progression of many cardiovascular disorders related to obesity. The results on respiratory sinus arrhythmia (RSA) magnitude changes as a parasympathetic index were not straightforward in previous studies on young obese subjects. Considering the potentially unbalanced ANS regulation with impaired parasympathetic control in obese patients, the aim of this study was to compare the relative contribution of baroreflex and non-baroreflex (central) mechanisms to the origin of RSA in obese vs. control subjects. To this end, we applied a recently proposed information-theoretic methodology - partial information decomposition (PID) - to the time series of heart rate variability (HRV, computed from RR intervals in the ECG), systolic blood pressure (SBP) variability, and respiration (RESP) pattern measured in 29 obese and 29 age- and gender-matched non-obese adolescents and young adults monitored in the resting supine position and during postural and cognitive stress evoked by head-up tilt and mental arithmetic. PID was used to quantify the so-called unique information transferred from RESP to HRV and from SBP to HRV, reflecting, respectively, non-baroreflex and RESP-unrelated baroreflex HRV mechanisms, and the redundant information transferred from (RESP, SBP) to HRV, reflecting RESP-related baroreflex RSA mechanisms. Our results suggest that obesity is associated: (i) with blunted involvement of non-baroreflex RSA mechanisms, documented by the lower unique information transferred from RESP to HRV at rest; and (ii) with a reduced response to postural stress (but not to mental stress), documented by the lack of changes in the unique information transferred from RESP and SBP to HRV in obese subjects moving from supine to upright, and by a decreased redundant information transfer in obese compared to controls in the upright position. These findings were observed in the presence of an unchanged RSA magnitude measured as the high frequency (HF) power of HRV, thus suggesting that the changes in ANS imbalance related to obesity in adolescents and young adults are subtle and can be revealed by dissecting RSA mechanisms into its components during various challenges.

Multivariate and Multiscale Complexity of Long-Range Correlated Cardiovascular and Respiratory Variability Series

Assessing the dynamical complexity of biological time series represents an important topic with potential applications ranging from the characterization of physiological states and pathological conditions to the calculation of diagnostic parameters. In particular, cardiovascular time series exhibit a variability produced by different physiological control mechanisms coupled with each other, which take into account several variables and operate across multiple time scales that result in the coexistence of short term dynamics and long-range correlations. The most widely employed technique to evaluate the dynamical complexity of a time series at different time scales, the so-called multiscale entropy (MSE), has been proven to be unsuitable in the presence of short multivariate time series to be analyzed at long time scales. This work aims at overcoming these issues via the introduction of a new method for the assessment of the multiscale complexity of multivariate time series. The method first exploits vector autoregressive fractionally integrated (VARFI) models to yield a linear parametric representation of vector stochastic processes characterized by short- and long-range correlations. Then, it provides an analytical formulation, within the theory of state-space models, of how the VARFI parameters change when the processes are observed across multiple time scales, which is finally exploited to derive MSE measures relevant to the overall multivariate process or to one constituent scalar process. The proposed approach is applied on cardiovascular and respiratory time series to assess the complexity of the heart period, systolic arterial pressure and respiration variability measured in a group of healthy subjects during conditions of postural and mental stress. Our results document that the proposed methodology can detect physiologically meaningful multiscale patterns of complexity documented previously, but can also capture significant variations in complexity which cannot be observed using standard methods that do not take into account long-range correlations.

Generalised Measures of Multivariate Information Content

The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of multivariate information content that can be accurately depicted using Venn diagrams for any number of random variables. These measures complement the existing measures of multivariate mutual information and are constructed by considering the algebraic structure of information sharing. It is shown that the distinct ways in which a set of marginal observers can share their information with a non-observing third party corresponds to the elements of a free distributive lattice. The redundancy lattice from partial information decomposition is then subsequently and independently derived by combining the algebraic structures of joint and shared information content.

Detecting Causality in Multivariate Time Series via Non-Uniform Embedding

Causal analysis based on non-uniform embedding schemes is an important way to detect the underlying interactions between dynamic systems. However, there are still some obstacles to estimating high-dimensional conditional mutual information and forming optimal mixed embedding vector in traditional non-uniform embedding schemes. In this study, we present a new non-uniform embedding method framed in information theory to detect causality for multivariate time series, named LM-PMIME, which integrates the low-dimensional approximation of conditional mutual information and the mixed search strategy for the construction of the mixed embedding vector. We apply the proposed method to simulations of linear stochastic, nonlinear stochastic, and chaotic systems, demonstrating its superiority over partial conditional mutual information from mixed embedding (PMIME) method. Moreover, the proposed method works well for multivariate time series with weak coupling strengths, especially for chaotic systems. In the actual application, we show its applicability to epilepsy multichannel electrocorticographic recordings.

Restricted information in a two-step cascade

A cell must sense extracellular and intracellular fluctuations and respond appropriately to survive for optimal cellular functioning. Accordingly, a cell builds up biochemical networks which can transduce information of extracellular and intracellular fluctuations accurately. We consider a generic two-step cascade as a model gene regulatory network containing three regulatory proteins S, X, and Y connected as S→X→Y. The intermediate node X is a stochastic variable, acts as an obstacle, and impedes the information flow from S to Y. We quantify the information that is restricted by X using the tools of information theory and term this as restricted information. In this context, we further propose two measurable quantities, restricted efficiency and information transfer efficiency. The former determines how efficiently X restricts the upstream information coming from S, while the latter computes the efficiency of X to pass the upstream information toward Y. We also quantify the information that is being uniquely transferred from X to Y, which determines the extent of the ability of X to act as a source of information. Our analysis shows that when the signal strength (or mean population of S, 〈s〉) is low, the intermediate X can carry forward the upstream information reliably as well, as it acts as a better source of information, thereby increasing the fidelity of the network. But at the high signal strength, X restricts most of the upstream information, and its ability to act as a source of information gets reduced. This leads to a loss of fidelity of the network.

A Study of Brain Neuronal and Functional Complexities Estimated Using Multiscale Entropy in Healthy Young Adults

Brain complexity estimated using sample entropy and multiscale entropy (MSE) has recently gained much attention to compare brain function between diseased or neurologically impaired groups and healthy control groups. Using resting-state functional magnetic resonance imaging (rfMRI) blood oxygen-level dependent (BOLD) signals in a large cohort (n = 967) of healthy young adults, the present study maps neuronal and functional complexities estimated by using MSE of BOLD signals and BOLD phase coherence connectivity, respectively, at various levels of the brain’s organization. The functional complexity explores patterns in a higher dimension than neuronal complexity and may better discern changes in brain functioning. The leave-one-subject-out cross-validation method is used to predict fluid intelligence using neuronal and functional complexity MSE values as features. While a wide range of scales was selected with neuronal complexity, only the first three scales were selected with functional complexity. Fewer scales are advantageous as they preclude the need for long BOLD signals to calculate good estimates of MSE. The presented results corroborate with previous findings and provide a baseline for other studies exploring the use of MSE to examine changes in brain function related to aging, diseases, and clinical disorders.

MAXENT3D_PID: An Estimator for the Maximum-Entropy Trivariate Partial Information Decomposition

Partial information decomposition (PID) separates the contributions of sources about a target into unique, redundant, and synergistic components of information. In essence, PID answers the question of “who knows what” of a system of random variables and hence has applications to a wide spectrum of fields ranging from social to biological sciences. The paper presents MaxEnt3D_Pid, an algorithm that computes the PID of three sources, based on a recently-proposed maximum entropy measure, using convex optimization (cone programming). We describe the algorithm and its associated software utilization and report the results of various experiments assessing its accuracy. Moreover, the paper shows that a hierarchy of bivariate and trivariate PID allows obtaining the finer quantities of the trivariate partial information measure.

Quantifying high-order interdependencies via multivariate extensions of the mutual information

This paper introduces a model-agnostic approach to study statistical synergy, a form of emergence in which patterns at large scales are not traceable from lower scales. Our framework leverages various multivariate extensions of Shannon's mutual information, and introduces the O-information as a metric that is capable of characterizing synergy- and redundancy-dominated systems. The O-information is a symmetric quantity, and can assess intrinsic properties of a system without dividing its parts into "predictors" and "targets." We develop key analytical properties of the O-information, and study how it relates to other metrics of high-order interactions from the statistical mechanics and neuroscience literature. Finally, as a proof of concept, we present an exploration on the relevance of statistical synergy in Baroque music scores.

Time, frequency and information domain analysis of short-term heart rate variability before and after focal and generalized seizures in epileptic children

OBJECTIVE

In this work we explore the potential of combining standard time and frequency domain indexes with novel information measures, to characterize pre- and post-ictal heart rate variability (HRV) in epileptic children, with the aim of differentiating focal and generalized epilepsy regarding the autonomic control mechanisms.

APPROACH

We analyze short-term HRV in 37 children suffering from generalized or focal epilepsy, monitored 10 s, 300 s, 600 s and 1800 s both before and after seizure episodes. Nine indexes are computed in time (mean, standard deviation of normal-to-normal intervals, root mean square of the successive differences (RMSSD)), frequency (low-to-high frequency power ratio LF/HF, normalized LF and HF power) and information (entropy, conditional entropy and self-entropy) domains. Focal and generalized epilepsy are compared through statistical analysis of the indexes and using linear discriminant analysis (LDA).

MAIN RESULTS

In children with focal epilepsy, early post-ictal phase is characterized by significant tachycardia, depressed HRV, increased LF power and LF/HF, and decreased complexity, progressively recovered across time windows after the episodes. Children with generalized seizures instead show significant tachycardia, lower RMSSD, higher LF power and LF/HF ratio before the seizure. These different behaviors are exploited by LDA analysis to separate focal and generalized epilepsy up to an accuracy of 75%. Results suggest a shift of the sympatho-vagal balance towards sympathetic dominance and vagal withdrawal, noticeable just after the termination of seizure episodes and then reverted in focal epilepsy, and persistent during inter-ictal and pre-ictal periods in generalized epilepsy.

SIGNIFICANCE

Our analysis helps in elucidating the pathophysiology of inter-ictal HRV autonomic control and the differential diagnosis of generalized and focal epilepsy. These findings may have clinical relevance since altered sympatho-vagal control can be related to a higher danger of morbidity and mortality, may reduce thresholds for life-threatening arrhythmias, and could be a biomarker of risk for sudden unexpected death in epilepsy.

A Parsimonious Granger Causality Formulation for Capturing Arbitrarily Long Multivariate Associations

High-frequency neuroelectric signals like electroencephalography (EEG) or magnetoencephalography (MEG) provide a unique opportunity to infer causal relationships between local activity of brain areas. While causal inference is commonly performed through classical Granger causality (GC) based on multivariate autoregressive models, this method may encounter important limitations (e.g., data paucity) in the case of high dimensional data from densely connected systems like the brain. Additionally, physiological signals often present long-range dependencies which commonly require high autoregressive model orders/number of parameters. We present a generalization of autoregressive models for GC estimation based on Wiener–Volterra decompositions with Laguerre polynomials as basis functions. In this basis, the introduction of only one additional global parameter allows to capture arbitrary long dependencies without increasing model order, hence retaining model simplicity, linearity and ease of parameters estimation. We validate our method in synthetic data generated from families of complex, densely connected networks and demonstrate superior performance as compared to classical GC. Additionally, we apply our framework to studying the directed human brain connectome through MEG data from 89 subjects drawn from the Human Connectome Project (HCP) database, showing that it is able to reproduce current knowledge as well as to uncover previously unknown directed influences between cortical and limbic brain regions.

Multivariate information processing characterizes fitness of a cascaded gene-transcription machinery

We report that a genetic two-step activation cascade processes diverse flavors of information, e.g., synergy, redundancy, and unique information. Our computations measuring reduction in Shannon entropies and reduction in variances produce differently behaving absolute magnitudes of these informational flavors. We find that similarity can be brought in if these terms are evaluated in fractions with respect to corresponding total information. Each of the input signal and final gene-product is found to generate common or redundant information fractions (mostly) to predict each other, whereas they also complement one another to harness synergistic information fraction, predicting the intermediate biochemical species. For an optimally growing signal to maintain fixed steady-state abundance of activated downstream gene-products, the interaction information fractions for this cascade module shift from net-redundancy to information-independence.

Multiscale Information Decomposition Dissects Control Mechanisms of Heart Rate Variability at Rest and During Physiological Stress

Heart rate variability (HRV; variability of the RR interval of the electrocardiogram) results from the activity of several coexisting control mechanisms, which involve the influence of respiration (RESP) and systolic blood pressure (SBP) oscillations operating across multiple temporal scales and changing in different physiological states. In this study, multiscale information decomposition is used to dissect the physiological mechanisms related to the genesis of HRV in 78 young volunteers monitored at rest and during postural and mental stress evoked by head-up tilt (HUT) and mental arithmetics (MA). After representing RR, RESP and SBP at different time scales through a recently proposed method based on multivariate state space models, the joint information transfer TRESP,SBP→RR is decomposed into unique, redundant and synergistic components, describing the strength of baroreflex modulation independent of respiration (USBP→RR), nonbaroreflex (URESP→RR) and baroreflex-mediated (RRESP,SBP→RR) respiratory influences, and simultaneous presence of baroreflex and nonbaroreflex respiratory influences (SRESP,SBP→RR), respectively. We find that fast (short time scale) HRV oscillations—respiratory sinus arrhythmia—originate from the coexistence of baroreflex and nonbaroreflex (central) mechanisms at rest, with a stronger baroreflex involvement during HUT. Focusing on slower HRV oscillations, the baroreflex origin is dominant and MA leads to its higher involvement. Respiration influences independent on baroreflex are present at long time scales, and are enhanced during HUT.

Electrophysiological Brain Connectivity: Theory and Implementation

We review the theory and algorithms of electrophysiological brain connectivity analysis. This tutorial is aimed at providing an introduction to brain functional connectivity from electrophysiological signals, including electroencephalography (EEG), magnetoencephalography (MEG), electrocorticography (ECoG), stereoelectroencephalography (SEEG). Various connectivity estimators are discussed, and algorithms introduced. Important issues for estimating and mapping brain functional connectivity with electrophysiology are discussed.

Information Dynamics of the Brain, Cardiovascular and Respiratory Network during Different Levels of Mental Stress

In this study, an analysis of brain, cardiovascular and respiratory dynamics was conducted combining information-theoretic measures with the Network Physiology paradigm during different levels of mental stress. Starting from low invasive recordings of electroencephalographic, electrocardiographic, respiratory, and blood volume pulse signals, the dynamical activity of seven physiological systems was probed with one-second time resolution measuring the time series of the δ, θ, α and β brain wave amplitudes, the cardiac period (RR interval), the respiratory amplitude, and the duration of blood pressure wave propagation (pulse arrival time, PAT). Synchronous 5-min windows of these time series, obtained from 18 subjects during resting wakefulness (REST), mental stress induced by mental arithmetic (MA) and sustained attention induced by serious game (SG), were taken to describe the dynamics of the nodes composing the observed physiological network. Network activity and connectivity were then assessed in the framework of information dynamics computing the new information generated by each node, the information dynamically stored in it, and the information transferred to it from the other network nodes. Moreover, the network topology was investigated using directed measures of conditional information transfer and assessing their statistical significance. We found that all network nodes dynamically produce and store significant amounts of information, with the new information being prevalent in the brain systems and the information storage being prevalent in the peripheral systems. The transition from REST to MA was associated with an increase of the new information produced by the respiratory signal time series (RESP), and that from MA to SG with a decrease of the new information produced by PAT. Each network node received a significant amount of information from the other nodes, with the highest amount transferred to RR and the lowest transferred to δ, θ, α and β. The topology of the physiological network underlying such information transfer was node- and state-dependent, with the peripheral subnetwork showing interactions from RR to PAT and between RESP and RR, PAT consistently across states, the brain subnetwork resulting more connected during MA, and the subnetwork of brain–peripheral interactions involving different brain rhythms in the three states and resulting primarily activated during MA. These results have both physiological relevance as regards the interpretation of central and autonomic effects on cardiovascular and respiratory variability, and practical relevance as regards the identification of features useful for the automatic distinction of different mental states.

Multiscale information storage of linear long-range correlated stochastic processes

Information storage, reflecting the capability of a dynamical system to keep predictable information during its evolution over time, is a key element of intrinsic distributed computation, useful for the description of the dynamical complexity of several physical and biological processes. Here we introduce a parametric approach which allows one to compute information storage across multiple timescales in stochastic processes displaying both short-term dynamics and long-range correlations (LRC). Our analysis is performed in the popular framework of multiscale entropy, whereby a time series is first "coarse grained" at the chosen timescale through low-pass filtering and downsampling, and then its complexity is evaluated in terms of conditional entropy. Within this framework, our approach makes use of linear fractionally integrated autoregressive (ARFI) models to derive analytical expressions for the information storage computed at multiple timescales. Specifically, we exploit state space models to provide the representation of lowpass filtered and downsampled ARFI processes, from which information storage is computed at any given timescale relating the process variance to the prediction error variance. This enhances the practical usability of multiscale information storage, as it enables a computationally reliable quantification of a complexity measure which incorporates the effects of LRC together with that of short-term dynamics. The proposed measure is first assessed in simulated ARFI processes reproducing different types of autoregressive dynamics and different degrees of LRC, studying both the theoretical values and the finite sample performance. We find that LRC alter substantially the complexity of ARFI processes even at short timescales, and that reliable estimation of complexity can be achieved at longer timescales only when LRC are properly modeled. Then, we assess multiscale information storage in physiological time series measured in humans during resting state and postural stress, revealing unprecedented responses to stress of the complexity of heart period and systolic arterial pressure variability, which are related to the different role played by LRC in the two conditions.

Multiscale Decomposition of Cardiovascular and Cardiorespiratory Information Transfer under General Anesthesia

The analysis of short-term cardiovascular and cardiorespiratory regulation during altered conscious states, such as those induced by anesthesia, requires to employ time series analysis methods able to deal with the multivariate and multiscale nature of the observed dynamics. To meet this requirement, the present study exploits the extension to multiscale analysis of recently proposed information decomposition methods which allow to quantify, from short realizations, the amounts of joint, unique, redundant and synergistic information transferred within multivariate time series. These methods were applied to the spontaneous variability of heart period (HP), systolic arterial pressure (SAP) and respiration (RESP) in patients undergoing coronary artery bypass graft monitored before and after the induction of general anesthesia. We found that, after anesthesia induction, information is processed within the cardiovascular network in a scale-dependent way: at short time scales, a shift from synergistic to redundant information transferred from SAP and RESP to HP occurs, which is associated with enhanced baroreflex-mediated respiratory effects on arterial pressure; at longer time scales, the increased information transfer from SAP to HP denotes an enhancement of the baroreflex coupling related to slow cardiovascular oscillations.

Interplay of synergy and redundancy in diamond motif

The formalism of partial information decomposition provides a number of independent components which altogether constitute the total information provided by the source variable(s) about the target variable(s). These non-overlapping terms are recognized as unique information, synergistic information, and redundant information. The metric of net synergy conceived as the difference between synergistic and redundant information is capable of detecting effective synergy, effective redundancy, and information independence among stochastic variables. The net synergy can be quantified using appropriate combinations of different Shannon mutual information terms. The utilization of the net synergy in network motifs with the nodes representing different biochemical species, involved in information sharing, uncovers rich store for exciting results. In the current study, we use this formalism to obtain a comprehensive understanding of the relative information processing mechanism in a diamond motif and two of its sub-motifs, namely, bifurcation and integration motif embedded within the diamond motif. The emerging patterns of effective synergy and effective redundancy and their contribution toward ensuring high fidelity information transmission are duly compared in the sub-motifs. Investigation on the metric of net synergy in independent bifurcation and integration motifs are also executed. In all of these computations, the crucial roles played by various systemic time scales, activation coefficients, and signal integration mechanisms at the output of the network topologies are especially emphasized. Following this plan of action, we become confident that the origin of effective synergy and effective redundancy can be architecturally justified by decomposing a diamond motif into bifurcation and integration motif. According to our conjecture, the presence of a common source of fluctuations creates effective redundancy. Our calculations reveal that effective redundancy empowers signal fidelity. Moreover, to achieve this, input signaling species avoids strong interaction with downstream intermediates. This strategy is capable of making the diamond motif noise-tolerant. Apart from the topological features, our study also puts forward the active contribution of additive and multiplicative signal integration mechanisms to nurture effective redundancy and effective synergy.

Time-varying MVAR algorithms for directed connectivity analysis: Critical comparison in simulations and benchmark EEG data

Human brain function depends on directed interactions between multiple areas that evolve in the subsecond range. Time-varying multivariate autoregressive (tvMVAR) modeling has been proposed as a way to help quantify directed functional connectivity strengths with high temporal resolution. While several tvMVAR approaches are currently available, there is a lack of unbiased systematic comparative analyses of their performance and of their sensitivity to parameter choices. Here, we critically compare four recursive tvMVAR algorithms and assess their performance while systematically varying adaptation coefficients, model order, and signal sampling rate. We also compared two ways of exploiting repeated observations: single-trial modeling followed by averaging, and multi-trial modeling where one tvMVAR model is fitted across all trials. Results from numerical simulations and from benchmark EEG recordings showed that: i) across a broad range of model orders all algorithms correctly reproduced patterns of interactions; ii) signal downsampling degraded connectivity estimation accuracy for most algorithms, although in some cases downsampling was shown to reduce variability in the estimates by lowering the number of parameters in the model; iii) single-trial modeling followed by averaging showed optimal performance with larger adaptation coefficients than previously suggested, and showed slower adaptation speeds than multi-trial modeling. Overall, our findings identify strengths and weaknesses of existing tvMVAR approaches and provide practical recommendations for their application to modeling dynamic directed interactions from electrophysiological signals.

Information Decomposition of Target Effects from Multi-Source Interactions: Perspectives on Previous, Current and Future Work

The formulation of the Partial Information Decomposition (PID) framework by Williams and Beer in 2010 attracted a significant amount of attention to the problem of defining redundant (or shared), unique and synergistic (or complementary) components of mutual information that a set of source variables provides about a target. This attention resulted in a number of measures proposed to capture these concepts, theoretical investigations into such measures, and applications to empirical data (in particular to datasets from neuroscience). In this Special Issue on “Information Decomposition of Target Effects from Multi-Source Interactions” at Entropy, we have gathered current work on such information decomposition approaches from many of the leading research groups in the field. We begin our editorial by providing the reader with a review of previous information decomposition research, including an overview of the variety of measures proposed, how they have been interpreted and applied to empirical investigations. We then introduce the articles included in the special issue one by one, providing a similar categorisation of these articles into: i. proposals of new measures; ii. theoretical investigations into properties and interpretations of such approaches, and iii. applications of these measures in empirical studies. We finish by providing an outlook on the future of the field.

Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices

What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example.