Leveraging information storage to select forecast-optimal parameters for delay-coordinate reconstructions

Causal evidence for cholinergic stabilization of attractor landscape dynamics

There is substantial evidence that neuromodulatory systems critically influence brain state dynamics; however, most work has been purely descriptive. Here, we quantify, using data combining local inactivation of the basal forebrain with simultaneous measurement of resting-state fMRI activity in the macaque, the causal role of long-range cholinergic input to the stabilization of brain states in the cerebral cortex. Local inactivation of the nucleus basalis of Meynert (nbM) leads to a decrease in the energy barriers required for an fMRI state transition in cortical ongoing activity. Moreover, the inactivation of particular nbM sub-regions predominantly affects information transfer in cortical regions known to receive direct anatomical projections. We demonstrate these results in a simple neurodynamical model of cholinergic impact on neuronal firing rates and slow hyperpolarizing adaptation currents. We conclude that the cholinergic system plays a critical role in stabilizing macroscale brain state dynamics.

The non-specific matrix thalamus facilitates the cortical information processing modes relevant for conscious awareness

The neurobiological mechanisms of arousal and anesthesia remain poorly understood. Recent evidence highlights the key role of interactions between the cerebral cortex and the diffusely projecting matrix thalamic nuclei. Here, we interrogate these processes in a whole-brain corticothalamic neural mass model endowed with targeted and diffusely projecting thalamocortical nuclei inferred from empirical data. This model captures key features seen in propofol anesthesia, including diminished network integration, lowered state diversity, impaired susceptibility to perturbation, and decreased corticocortical coherence. Collectively, these signatures reflect a suppression of information transfer across the cerebral cortex. We recover these signatures of conscious arousal by selectively stimulating the matrix thalamus, recapitulating empirical results in macaque, as well as wake-like information processing states that reflect the thalamic modulation of large-scale cortical attractor dynamics. Our results highlight the role of matrix thalamocortical projections in shaping many features of complex cortical dynamics to facilitate the unique communication states supporting conscious awareness.

Early lock-in of structured and specialised information flows during neural development

The brains of many organisms are capable of complicated distributed computation underpinned by a highly advanced information processing capacity. Although substantial progress has been made towards characterising the information flow component of this capacity in mature brains, there is a distinct lack of work characterising its emergence during neural development. This lack of progress has been largely driven by the lack of effective estimators of information processing operations for spiking data. Here, we leverage recent advances in this estimation task in order to quantify the changes in transfer entropy during development. We do so by studying the changes in the intrinsic dynamics of the spontaneous activity of developing dissociated neural cell cultures. We find that the quantity of information flowing across these networks undergoes a dramatic increase across development. Moreover, the spatial structure of these flows exhibits a tendency to lock-in at the point when they arise. We also characterise the flow of information during the crucial periods of population bursts. We find that, during these bursts, nodes tend to undertake specialised computational roles as either transmitters, mediators, or receivers of information, with these roles tending to align with their average spike ordering. Further, we find that these roles are regularly locked-in when the information flows are established. Finally, we compare these results to information flows in a model network developing according to a spike-timing-dependent plasticity learning rule. Similar temporal patterns in the development of information flows were observed in these networks, hinting at the broader generality of these phenomena.

Toward automated extraction and characterization of scaling regions in dynamical systems

Scaling regions-intervals on a graph where the dependent variable depends linearly on the independent variable-abound in dynamical systems, notably in calculations of invariants like the correlation dimension or a Lyapunov exponent. In these applications, scaling regions are generally selected by hand, a process that is subjective and often challenging due to noise, algorithmic effects, and confirmation bias. In this paper, we propose an automated technique for extracting and characterizing such regions. Starting with a two-dimensional plot-e.g., the values of the correlation integral, calculated using the Grassberger-Procaccia algorithm over a range of scales-we create an ensemble of intervals by considering all possible combinations of end points, generating a distribution of slopes from least squares fits weighted by the length of the fitting line and the inverse square of the fit error. The mode of this distribution gives an estimate of the slope of the scaling region (if it exists). The end points of the intervals that correspond to the mode provide an estimate for the extent of that region. When there is no scaling region, the distributions will be wide and the resulting error estimates for the slope will be large. We demonstrate this method for computations of dimension and Lyapunov exponent for several dynamical systems and show that it can be useful in selecting values for the parameters in time-delay reconstructions.

TimeCycle: Topology Inspired MEthod for the Detection of Cycling Transcripts in Circadian Time-Series Data

MOTIVATION

The circadian rhythm drives the oscillatory expression of thousands of genes across all tissues. The recent revolution in high-throughput transcriptomics, coupled with the significant implications of the circadian clock for human health, has sparked an interest in circadian profiling studies to discover genes under circadian control.

RESULT

We present TimeCycle: a topology-based rhythm detection method designed to identify cycling transcripts. For a given time-series, the method reconstructs the state space using time-delay embedding, a data transformation technique from dynamical systems theory. In the embedded space, Takens' theorem proves that the dynamics of a rhythmic signal will exhibit circular patterns. The degree of circularity of the embedding is calculated as a persistence score using persistent homology, an algebraic method for discerning the topological features of data. By comparing the persistence scores to a bootstrapped null distribution, cycling genes are identified. Results in both synthetic and biological data highlight TimeCycle's ability to identify cycling genes across a range of sampling schemes, number of replicates, and missing data. Comparison to competing methods highlights their relative strengths, providing guidance as to the optimal choice of cycling detection method.

AVAILABILITY

A fully documented open-source R package implementing TimeCycle is available at: https://nesscoder.github.io/TimeCycle/.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Using curvature to select the time lag for delay reconstruction

We propose a curvature-based approach for choosing a good value for the time-delay parameter τ in delay reconstructions. The idea is based on the effects of the delay on the geometry of the reconstructions. If the delay is too small, the reconstructed dynamics are flattened along the main diagonal of the embedding space; too-large delays, on the other hand, can overfold the dynamics. Calculating the curvature of a two-dimensional delay reconstruction is an effective way to identify these extremes and to find a middle ground between them: both the sharp reversals at the extremes of an insufficiently unfolded reconstruction and the bends in an overfolded one create spikes in the curvature. We operationalize this observation by computing the mean Menger curvature of a trajectory segment on 2D reconstructions as a function of time delay. We show that the minimum of these values gives an effective heuristic for choosing the time delay. In addition, we show that this curvature-based heuristic is useful even in cases where the customary approach, which uses average mutual information, fails-e.g., noisy or filtered data.

Deriving pairwise transfer entropy from network structure and motifs

Transfer entropy (TE) is an established method for quantifying directed statistical dependencies in neuroimaging and complex systems datasets. The pairwise (or bivariate) TE from a source to a target node in a network does not depend solely on the local source-target link weight, but on the wider network structure that the link is embedded in. This relationship is studied using a discrete-time linearly coupled Gaussian model, which allows us to derive the TE for each link from the network topology. It is shown analytically that the dependence on the directed link weight is only a first approximation, valid for weak coupling. More generally, the TE increases with the in-degree of the source and decreases with the in-degree of the target, indicating an asymmetry of information transfer between hubs and low-degree nodes. In addition, the TE is directly proportional to weighted motif counts involving common parents or multiple walks from the source to the target, which are more abundant in networks with a high clustering coefficient than in random networks. Our findings also apply to Granger causality, which is equivalent to TE for Gaussian variables. Moreover, similar empirical results on random Boolean networks suggest that the dependence of the TE on the in-degree extends to nonlinear dynamics.

Transitions in information processing dynamics at the whole-brain network level are driven by alterations in neural gain

A key component of the flexibility and complexity of the brain is its ability to dynamically adapt its functional network structure between integrated and segregated brain states depending on the demands of different cognitive tasks. Integrated states are prevalent when performing tasks of high complexity, such as maintaining items in working memory, consistent with models of a global workspace architecture. Recent work has suggested that the balance between integration and segregation is under the control of ascending neuromodulatory systems, such as the noradrenergic system, via changes in neural gain (in terms of the amplification and non-linearity in stimulus-response transfer function of brain regions). In a previous large-scale nonlinear oscillator model of neuronal network dynamics, we showed that manipulating neural gain parameters led to a 'critical' transition in phase synchrony that was associated with a shift from segregated to integrated topology, thus confirming our original prediction. In this study, we advance these results by demonstrating that the gain-mediated phase transition is characterized by a shift in the underlying dynamics of neural information processing. Specifically, the dynamics of the subcritical (segregated) regime are dominated by information storage, whereas the supercritical (integrated) regime is associated with increased information transfer (measured via transfer entropy). Operating near to the critical regime with respect to modulating neural gain parameters would thus appear to provide computational advantages, offering flexibility in the information processing that can be performed with only subtle changes in gain control. Our results thus link studies of whole-brain network topology and the ascending arousal system with information processing dynamics, and suggest that the constraints imposed by the ascending arousal system constrain low-dimensional modes of information processing within the brain.

Information dynamics with confidence: Using reservoir computing to construct confidence intervals for information-dynamic measures

Information dynamics provides a broad set of measures for characterizing how a dynamical system stores, processes, and transmits information. While estimators for these measures are commonly used in applications, the statistical properties of these estimators for finite time series are not well understood. In particular, the precision of a given estimate is generally unknown. We develop confidence intervals for generic information-dynamic parameters using a bootstrap procedure. The bootstrap procedure uses an echo state network, a particular instance of a reservoir computer, as a simulator to generate bootstrap samples from a given time series. We perform a Monte Carlo analysis to investigate the performance of the bootstrap confidence intervals in terms of their coverage and expected lengths with two model systems and compare their performance to a simulator based on the random analog predictor. We find that our bootstrap procedure generates confidence intervals with nominal, or near nominal, coverage of the information-dynamic measures, with smaller expected length than the random analog predictor-based confidence intervals. Finally, we demonstrate the applicability of the confidence intervals for characterizing the information dynamics of a time series of sunspot counts.

Large-scale directed network inference with multivariate transfer entropy and hierarchical statistical testing

Network inference algorithms are valuable tools for the study of large-scale neuroimaging datasets. Multivariate transfer entropy is well suited for this task, being a model-free measure that captures nonlinear and lagged dependencies between time series to infer a minimal directed network model. Greedy algorithms have been proposed to efficiently deal with high-dimensional datasets while avoiding redundant inferences and capturing synergistic effects. However, multiple statistical comparisons may inflate the false positive rate and are computationally demanding, which limited the size of previous validation studies. The algorithm we present-as implemented in the IDTxl open-source software-addresses these challenges by employing hierarchical statistical tests to control the family-wise error rate and to allow for efficient parallelization. The method was validated on synthetic datasets involving random networks of increasing size (up to 100 nodes), for both linear and nonlinear dynamics. The performance increased with the length of the time series, reaching consistently high precision, recall, and specificity (>98% on average) for 10,000 time samples. Varying the statistical significance threshold showed a more favorable precision-recall trade-off for longer time series. Both the network size and the sample size are one order of magnitude larger than previously demonstrated, showing feasibility for typical EEG and magnetoencephalography experiments.

Conformity in the collective: differences in hunger affect individual and group behavior in a shoaling fish

Animal groups are often composed of individuals that vary according to behavioral, morphological, and internal state parameters. Understanding the importance of such individual-level heterogeneity to the establishment and maintenance of coherent group responses is of fundamental interest in collective behavior. We examined the influence of hunger on the individual and collective behavior of groups of shoaling fish, x-ray tetras (Pristella maxillaris). Fish were assigned to one of two nutritional states, satiated or hungry, and then allocated to 5 treatments that represented different ratios of satiated to hungry individuals (8 hungry, 8 satiated, 4:4 hungry:satiated, 2:6 hungry:satiated, 6:2 hungry:satiated). Our data show that groups with a greater proportion of hungry fish swam faster and exhibited greater nearest neighbor distances. Within groups, however, there was no difference in the swimming speeds of hungry versus well-fed fish, suggesting that group members conform and adapt their swimming speed according to the overall composition of the group. We also found significant differences in mean group transfer entropy, suggesting stronger patterns of information flow in groups comprising all, or a majority of, hungry individuals. In contrast, we did not observe differences in polarization, a measure of group alignment, within groups across treatments. Taken together these results demonstrate that the nutritional state of animals within social groups impacts both individual and group behavior, and that members of heterogenous groups can adapt their behavior to facilitate coherent collective motion.

Cohesion, order and information flow in the collective motion of mixed-species shoals

Despite the frequency with which mixed-species groups are observed in nature, studies of collective behaviour typically focus on single-species groups. Here, we quantify and compare the patterns of interactions between three fish species, threespine sticklebacks (Gasterosteus aculeatus), ninespine sticklebacks (Pungitius pungitius) and roach (Rutilus rutilus) in both single- and mixed-species shoals in the laboratory. Pilot data confirmed that the three species form both single- and mixed-species shoals in the wild. In our laboratory study, we found that single-species groups were more polarized than mixed-species groups, while single-species groups of threespine sticklebacks and roach were more cohesive than mixed shoals of these species. Furthermore, while there was no difference between the inter-individual distances between threespine and ninespine sticklebacks within mixed-species groups, there was some evidence of segregation by species in mixed groups of threespine sticklebacks and roach. There were differences between treatments in mean pairwise transfer entropy, and in particular we identify species-differences in information use within the mixed-species groups, and, similarly, differences in responses to conspecifics and heterospecifics in mixed-species groups. We speculate that differences in the patterns of interactions between species in mixed-species groups may determine patterns of fission and fusion in such groups.

Characterizing information-theoretic storage and transfer in continuous time processes

The characterization of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modeling the intrinsic information processing conducted by a process represented as a time series, but to date has only been formulated in discrete time. Building on previous work which demonstrated how to formulate transfer entropy in continuous time, we give a total account of information processing in this setting, incorporating information storage. We find that a convergent rate of predictive capacity, comprising the transfer entropy and active information storage, does not exist, arising through divergent rates of active information storage. We identify that active information storage can be decomposed into two separate quantities that characterize predictive capacity stored in a process: active memory utilization and instantaneous predictive capacity. The latter involves prediction related to path regularity and so solely inherits the divergent properties of the active information storage, while the former permits definitions of pathwise and rate quantities. We formulate measures of memory utilization for jump and neural spiking processes and illustrate measures of information processing in synthetic neural spiking models and coupled Ornstein-Uhlenbeck models. The application to synthetic neural spiking models demonstrates that active memory utilization for point processes consists of discontinuous jump contributions (at spikes) interrupting a continuously varying contribution (relating to waiting times between spikes), complementing the behavior previously demonstrated for transfer entropy in these processes.

Information-theoretic model selection for optimal prediction of stochastic dynamical systems from data

In the absence of mechanistic or phenomenological models of real-world systems, data-driven models become necessary. The discovery of various embedding theorems in the 1980s and 1990s motivated a powerful set of tools for analyzing deterministic dynamical systems via delay-coordinate embeddings of observations of their component states. However, in many branches of science, the condition of operational determinism is not satisfied, and stochastic models must be brought to bear. For such stochastic models, the tool set developed for delay-coordinate embedding is no longer appropriate, and a new toolkit must be developed. We present an information-theoretic criterion, the negative log-predictive likelihood, for selecting the embedding dimension for a predictively optimal data-driven model of a stochastic dynamical system. We develop a nonparametric estimator for the negative log-predictive likelihood and compare its performance to a recently proposed criterion based on active information storage. Finally, we show how the output of the model selection procedure can be used to compare candidate predictors for a stochastic system to an information-theoretic lower bound.

Incomplete phase-space method to reveal time delay from scalar time series

A computationally quick and conceptually simple method to recover time delay of the chaotic system from scalar time series is developed in this paper. We show that the orbits in the incomplete two-dimensional reconstructed phase-space will show local clustering phenomenon after the component reordering procedure proposed in this work. We find that information captured by the incomplete two-dimensional reconstructed phase-space is related to the time delay τ_{0} present in the system, and will be transferred to the reordered component by the procedure of component reordering. We then propose the segmented mean variance (SMV) from the reordered component to identify the time delay τ_{0} of the system. The proposed SMV shows clear maximum when the embedding delay τ of the incomplete reconstruction matches the time delay τ_{0} of the chaotic system. Numerical data generated by a time-delay system based on the Mackey-Glass equation operating in the chaotic regime are used to illustrate the effectiveness of the proposed SMV. Experimental results show that the proposed SMV is robust to additive observational noise and is able to recover the time delay of the chaotic system even though the amount of data is relatively small and the feedback strength is weak. Moreover, the time complexity of the proposed method is quite low.