An Information-Theoretic Framework to Measure the Dynamic Interaction Between Neural Spike Trains

Testing dynamic correlations and nonlinearity in bivariate time series through information measures and surrogate data analysis

The increasing availability of time series data depicting the evolution of physical system properties has prompted the development of methods focused on extracting insights into the system behavior over time, discerning whether it stems from deterministic or stochastic dynamical systems. Surrogate data testing plays a crucial role in this process by facilitating robust statistical assessments. This ensures that the observed results are not mere occurrences by chance, but genuinely reflect the inherent characteristics of the underlying system. The initial process involves formulating a null hypothesis, which is tested using surrogate data in cases where assumptions about the underlying distributions are absent. A discriminating statistic is then computed for both the original data and each surrogate data set. Significantly deviating values between the original data and the surrogate data ensemble lead to the rejection of the null hypothesis. In this work, we present various surrogate methods designed to assess specific statistical properties in random processes. Specifically, we introduce methods for evaluating the presence of autodependencies and nonlinear dynamics within individual processes, using Information Storage as a discriminating statistic. Additionally, methods are introduced for detecting coupling and nonlinearities in bivariate processes, employing the Mutual Information Rate for this purpose. The surrogate methods introduced are first tested through simulations involving univariate and bivariate processes exhibiting both linear and nonlinear dynamics. Then, they are applied to physiological time series of Heart Period (RR intervals) and respiratory flow (RESP) variability measured during spontaneous and paced breathing. Simulations demonstrated that the proposed methods effectively identify essential dynamical features of stochastic systems. The real data application showed that paced breathing, at low breathing rate, increases the predictability of the individual dynamics of RR and RESP and dampens nonlinearity in their coupled dynamics.

Adaptive modeling and inference of higher-order coordination in neuronal assemblies: A dynamic greedy estimation approach

Central in the study of population codes, coordinated ensemble spiking activity is widely observable in neural recordings with hypothesized roles in robust stimulus representation, interareal communication, and learning and memory formation. Model-free measures of synchrony characterize coherent pairwise activity but not higher-order interactions, a limitation transcended by statistical models of ensemble spiking activity. However, existing model-based analyses often impose assumptions about the relevance of higher-order interactions and require repeated trials to characterize dynamics in the correlational structure of ensemble activity. To address these shortcomings, we propose an adaptive greedy filtering algorithm based on a discretized mark point-process model of ensemble spiking and a corresponding statistical inference framework to identify significant higher-order coordination. In the course of developing a precise statistical test, we show that confidence intervals can be constructed for greedily estimated parameters. We demonstrate the utility of our proposed methods on simulated neuronal assemblies. Applied to multi-electrode recordings from human and rat cortical assemblies, our proposed methods provide new insights into the dynamics underlying localized population activity during transitions between brain states.

Inference of network connectivity from temporally binned spike trains

BACKGROUND

Processing neural activity to reconstruct network connectivity is a central focus of neuroscience, yet the spatiotemporal requisites of biological nervous systems are challenging for current neuronal sensing modalities. Consequently, methods that leverage limited data to successfully infer synaptic connections, predict activity at single unit resolution, and decipher their effect on whole systems, can uncover critical information about neural processing. Despite the emergence of powerful methods for inferring connectivity, network reconstruction based on temporally subsampled data remains insufficiently unexplored.

NEW METHOD

We infer synaptic weights by processing firing rates within variable time bins for a heterogeneous feed-forward network of excitatory, inhibitory, and unconnected units. We assess classification and optimize model parameters for postsynaptic spike train reconstruction. We test our method on a physiological network of leaky integrate-and-fire neurons displaying bursting patterns and assess prediction of postsynaptic activity from microelectrode array data.

RESULTS

Results reveal parameters for improved prediction and performance and suggest that lower resolution data and limited access to neurons can be preferred.

COMPARISON WITH EXISTING METHOD(S)

Recent computational methods demonstrate highly improved reconstruction of connectivity from networks of parallel spike trains by considering spike lag, time-varying firing rates, and other underlying dynamics. However, these methods insufficiently explore temporal subsampling representative of novel data types.

CONCLUSIONS

We provide a framework for reverse engineering neural networks from data with limited temporal quality, describing optimal parameters for each bin size, which can be further improved using non-linear methods and applied to more complicated readouts and connectivity distributions in multiple brain circuits.

Decomposing the Mutual Information Rate of Heart Period and Respiration Variability Series to Assess Cardiorespiratory Interactions

Heart rate variability results from the coupled activity of the cardiovascular and cardiorespiratory systems, which have their own internal regulation mechanisms but also interact with each other and with the autonomic nervous system to maintain homeostasis. In this work, the assessment of these physiological mechanisms is carried out decomposing the Mutual Information Rate (MIR), an information-theoretic measure of the interdependence between coupled processes, into terms of entropy rate or conditional mutual information related respectively to complexity and causality measures. These measures are computed using a non-parametric approach based on nearest-neighbors. The proposed framework is first tested on simulated autoregressive processes and then applied to experimental data consisting of heart period and respiratory time series measured in healthy subjects monitored at rest and during head-up tilt. Our results evidence that MIR decomposition is able to highlight the interdependence of short-term physiological mechanisms of cardiorespiratory interactions during postural stress.

Exploring the Short-Term Memory of Heart Rate Variability through Model-Free Information Measures

In this work, we perform a comparative analysis of discrete- and continuous-time estimators of information-theoretic measures quantifying the concept of memory utilization in short-term heart rate variability (HRV). Specifically, considering heartbeat intervals in discrete time we compute the measure of information storage (IS) and decompose it into immediate memory utilization (IMU) and longer memory utilization (MU) terms; considering the timings of heartbeats in continuous time we compute the measure of MU rate (MUR). All measures are computed through model-free approaches based on nearest neighbor entropy estimators applied to the HRV series of a group of 15 healthy subjects measured at rest and during postural stress. We find, moving from rest to stress, statistically significant increases of the IS and the IMU, as well as of the MUR. Our results suggest that both discrete-time and continuous-time approaches can detect the higher predictive capacity of HRV occurring with postural stress, and that such increased memory utilization is due to fast mechanisms likely related to sympathetic activation.

Connectivity Analysis in EEG Data: A Tutorial Review of the State of the Art and Emerging Trends

Understanding how different areas of the human brain communicate with each other is a crucial issue in neuroscience. The concepts of structural, functional and effective connectivity have been widely exploited to describe the human connectome, consisting of brain networks, their structural connections and functional interactions. Despite high-spatial-resolution imaging techniques such as functional magnetic resonance imaging (fMRI) being widely used to map this complex network of multiple interactions, electroencephalographic (EEG) recordings claim high temporal resolution and are thus perfectly suitable to describe either spatially distributed and temporally dynamic patterns of neural activation and connectivity. In this work, we provide a technical account and a categorization of the most-used data-driven approaches to assess brain-functional connectivity, intended as the study of the statistical dependencies between the recorded EEG signals. Different pairwise and multivariate, as well as directed and non-directed connectivity metrics are discussed with a pros-cons approach, in the time, frequency, and information-theoretic domains. The establishment of conceptual and mathematical relationships between metrics from these three frameworks, and the discussion of novel methodological approaches, will allow the reader to go deep into the problem of inferring functional connectivity in complex networks. Furthermore, emerging trends for the description of extended forms of connectivity (e.g., high-order interactions) are also discussed, along with graph-theory tools exploring the topological properties of the network of connections provided by the proposed metrics. Applications to EEG data are reviewed. In addition, the importance of source localization, and the impacts of signal acquisition and pre-processing techniques (e.g., filtering, source localization, and artifact rejection) on the connectivity estimates are recognized and discussed. By going through this review, the reader could delve deeply into the entire process of EEG pre-processing and analysis for the study of brain functional connectivity and learning, thereby exploiting novel methodologies and approaches to the problem of inferring connectivity within complex networks.

Comparison of discretization strategies for the model-free information-theoretic assessment of short-term physiological interactions

This work presents a comparison between different approaches for the model-free estimation of information-theoretic measures of the dynamic coupling between short realizations of random processes. The measures considered are the mutual information rate (MIR) between two random processes X and Y and the terms of its decomposition evidencing either the individual entropy rates of X and Y and their joint entropy rate, or the transfer entropies from X to Y and from Y to X and the instantaneous information shared by X and Y. All measures are estimated through discretization of the random variables forming the processes, performed either via uniform quantization (binning approach) or rank ordering (permutation approach). The binning and permutation approaches are compared on simulations of two coupled non-identical Hènon systems and on three datasets, including short realizations of cardiorespiratory (CR, heart period and respiration flow), cardiovascular (CV, heart period and systolic arterial pressure), and cerebrovascular (CB, mean arterial pressure and cerebral blood flow velocity) measured in different physiological conditions, i.e., spontaneous vs paced breathing or supine vs upright positions. Our results show that, with careful selection of the estimation parameters (i.e., the embedding dimension and the number of quantization levels for the binning approach), meaningful patterns of the MIR and of its components can be achieved in the analyzed systems. On physiological time series, we found that paced breathing at slow breathing rates induces less complex and more coupled CR dynamics, while postural stress leads to unbalancing of CV interactions with prevalent baroreflex coupling and to less complex pressure dynamics with preserved CB interactions. These results are better highlighted by the permutation approach, thanks to its more parsimonious representation of the discretized dynamic patterns, which allows one to explore interactions with longer memory while limiting the curse of dimensionality.

Adaptive Spike-Like Representation of EEG Signals for Sleep Stages Scoring

Recently there has seen promising results on auto-matic stage scoring by extracting spatio-temporal features from electroencephalogram (EEG). Such methods entail laborious manual feature engineering and domain knowledge. In this study, we propose an adaptive scheme to probabilistically encode, filter and accumulate the input signals and weight the resultant features by the half-Gaussian probabilities of signal intensities. The adaptive representations are subsequently fed into a transformer model to automatically mine the relevance between features and corresponding stages. Extensive exper-iments on the largest public dataset against state-of-the-art methods validate the effectiveness of our proposed method and reveal promising future directions.

THPs: Topological Hawkes Processes for Learning Causal Structure on Event Sequences

Learning causal structure among event types on multitype event sequences is an important but challenging task. Existing methods, such as the Multivariate Hawkes processes, mostly assumed that each sequence is independent and identically distributed. However, in many real-world applications, it is commonplace to encounter a topological network behind the event sequences such that an event is excited or inhibited not only by its history but also by its topological neighbors. Consequently, the failure in describing the topological dependency among the event sequences leads to the error detection of the causal structure. By considering the Hawkes processes from the view of temporal convolution, we propose a topological Hawkes process (THP) to draw a connection between the graph convolution in the topology domain and the temporal convolution in time domains. We further propose a causal structure learning method on THP in a likelihood framework. The proposed method is featured with the graph convolution-based likelihood function of THP and a sparse optimization scheme with an Expectation-Maximization of the likelihood function. Theoretical analysis and experiments on both synthetic and real-world data demonstrate the effectiveness of the proposed method.

Early lock-in of structured and specialised information flows during neural development

The brains of many organisms are capable of complicated distributed computation underpinned by a highly advanced information processing capacity. Although substantial progress has been made towards characterising the information flow component of this capacity in mature brains, there is a distinct lack of work characterising its emergence during neural development. This lack of progress has been largely driven by the lack of effective estimators of information processing operations for spiking data. Here, we leverage recent advances in this estimation task in order to quantify the changes in transfer entropy during development. We do so by studying the changes in the intrinsic dynamics of the spontaneous activity of developing dissociated neural cell cultures. We find that the quantity of information flowing across these networks undergoes a dramatic increase across development. Moreover, the spatial structure of these flows exhibits a tendency to lock-in at the point when they arise. We also characterise the flow of information during the crucial periods of population bursts. We find that, during these bursts, nodes tend to undertake specialised computational roles as either transmitters, mediators, or receivers of information, with these roles tending to align with their average spike ordering. Further, we find that these roles are regularly locked-in when the information flows are established. Finally, we compare these results to information flows in a model network developing according to a spike-timing-dependent plasticity learning rule. Similar temporal patterns in the development of information flows were observed in these networks, hinting at the broader generality of these phenomena.

Feedforward and feedback influences through distinct frequency bands between two spiking-neuron networks

Several studies on brain signals suggested that bottom-up and top-down influences are exerted through distinct frequency bands among visual cortical areas. It was recently shown that theta and gamma rhythms subserve feedforward, whereas the feedback influence is dominated by the alpha-beta rhythm in primates. A few theoretical models for reproducing these effects have been proposed so far. Here we show that a simple but biophysically plausible two-network motif composed of spiking-neuron models and chemical synapses can exhibit feedforward and feedback influences through distinct frequency bands. Different from previous studies, this kind of model allows us to study directed influences not only at the population level, by using a proxy for the local field potential, but also at the cellular level, by using the neuronal spiking series.

Measuring the Rate of Information Transfer in Point-Process Data: Application to Cardiovascular Interactions

We present the implementation to cardiovascular variability of a method for the information-theoretic estimation of the directed interactions between event-based data. The method allows to compute the transfer entropy rate (TER) from a source to a target point process in continuous time, thus overcoming the severe limitations associated with time discretization of event-based processes. In this work, the method is evaluated on coupled cardiovascular point processes representing the heartbeat dynamics and the related peripheral pulsation, first using a physiologically-based simulation model and then studying real point-process data from healthy subjects monitored at rest and during postural stress. Our results document the ability of TER to detect direction and strength of the interactions between cardiovascular processes, also highlighting physiologically plausible interaction mechanisms.

Relationship between mutual information and cross-correlation time scale of observability as measures of connectivity strength

The task of identifying and characterizing network structures out of experimentally observed time series is tackled by implementing different solutions, ranging from entropy-based techniques to the evaluation of the significance of observed correlation estimators. Among the metrics that belong to the first class, mutual information is of major importance due to the relative simplicity of implementation and its relying on the crucial concept of entropy. With regard to the second class, a method that allows us to assess the connectivity strength of a link in terms of a time scale of its observability via the significance estimate of measured cross correlation was recently shown to provide a reliable tool to study network structures. In this paper, we investigate the relationship between this last metric and mutual information by simultaneously assessing both metrics on large sets of data extracted from three experimental contexts, human brain magnetoencephalography, human brain electroencephalography, and surface wind measurements carried out on a small regional scale, as well as on simulated coupled, auto-regressive processes. We show that the relationship is well described by a power law and provide a theoretical explanation based on a simple noise and signal model. Besides further upholding the reliability of cross-correlation time scale of observability, the results show that the combined use of this metric and mutual information can be used as a valuable tool to identify and characterize connectivity links in a wide range of experimental contexts.

Measuring the Rate of Information Exchange in Point-Process Data With Application to Cardiovascular Variability

The amount of information exchanged per unit of time between two dynamic processes is an important concept for the analysis of complex systems. Theoretical formulations and data-efficient estimators have been recently introduced for this quantity, known as the mutual information rate (MIR), allowing its continuous-time computation for event-based data sets measured as realizations of coupled point processes. This work presents the implementation of MIR for point process applications in Network Physiology and cardiovascular variability, which typically feature short and noisy experimental time series. We assess the bias of MIR estimated for uncoupled point processes in the frame of surrogate data, and we compensate it by introducing a corrected MIR (cMIR) measure designed to return zero values when the two processes do not exchange information. The method is first tested extensively in synthetic point processes including a physiologically-based model of the heartbeat dynamics and the blood pressure propagation times, where we show the ability of cMIR to compensate the negative bias of MIR and return statistically significant values even for weakly coupled processes. The method is then assessed in real point-process data measured from healthy subjects during different physiological conditions, showing that cMIR between heartbeat and pressure propagation times increases significantly during postural stress, though not during mental stress. These results document that cMIR reflects physiological mechanisms of cardiovascular variability related to the joint neural autonomic modulation of heart rate and arterial compliance.