Information storage, loop motifs, and clustered structure in complex networks

Perturbed Information Processing Complexity in Experimental Epilepsy

Comorbidities, such as cognitive deficits, which often accompany epilepsies, constitute a basal state, while seizures are rare and transient events. This suggests that neural dynamics, in particular those supporting cognitive function, are altered in a permanent manner in epilepsy. Here, we test the hypothesis that primitive processes of information processing at the core of cognitive function (i.e., storage and sharing of information) are altered in the hippocampus and the entorhinal cortex in experimental epilepsy in adult, male Wistar rats. We find that information storage and sharing are organized into substates across the stereotypic states of slow and theta oscillations in both epilepsy and control conditions. However, their internal composition and organization through time are disrupted in epilepsy, partially losing brain state selectivity compared with controls, and shifting toward a regimen of disorder. We propose that the alteration of information processing at this algorithmic level of computation, the theoretical intermediate level between structure and function, may be a mechanism behind the emergent and widespread comorbidities associated with epilepsy, and perhaps other disorders.SIGNIFICANCE STATEMENT Comorbidities, such as cognitive deficits, which often accompany epilepsies, constitute a basal state, while seizures are rare and transient events. This suggests that neural dynamics, in particular those supporting cognitive function, are altered in a permanent manner in epilepsy. Here, we show that basic processes of information processing at the core of cognitive function (i.e., storage and sharing of information) are altered in the hippocampus and the entorhinal cortex (two regions involved in memory processes) in experimental epilepsy. Such disruption of information processing at the algorithmic level itself could underlie the general performance impairments in epilepsy.

Searching for Key Cycles in a Complex Network

Searching for key nodes and edges in a network is a long-standing problem. Recently cycle structure in a network has received more attention. Is it possible to propose a ranking algorithm for cycle importance? We address the problem of identifying the key cycles of a network. First, we provide a more concrete definition of importance-in terms of Fiedler value (the second smallest Laplacian eigenvalue). Key cycles are those that contribute most substantially to the dynamical behavior of the network. Second, by comparing the sensitivity of Fiedler value to different cycles, a neat index for ranking cycles is provided. Numerical examples are given to show the effectiveness of this method.

Relating local connectivity and global dynamics in recurrent excitatory-inhibitory networks

How the connectivity of cortical networks determines the neural dynamics and the resulting computations is one of the key questions in neuroscience. Previous works have pursued two complementary approaches to quantify the structure in connectivity. One approach starts from the perspective of biological experiments where only the local statistics of connectivity motifs between small groups of neurons are accessible. Another approach is based instead on the perspective of artificial neural networks where the global connectivity matrix is known, and in particular its low-rank structure can be used to determine the resulting low-dimensional dynamics. A direct relationship between these two approaches is however currently missing. Specifically, it remains to be clarified how local connectivity statistics and the global low-rank connectivity structure are inter-related and shape the low-dimensional activity. To bridge this gap, here we develop a method for mapping local connectivity statistics onto an approximate global low-rank structure. Our method rests on approximating the global connectivity matrix using dominant eigenvectors, which we compute using perturbation theory for random matrices. We demonstrate that multi-population networks defined from local connectivity statistics for which the central limit theorem holds can be approximated by low-rank connectivity with Gaussian-mixture statistics. We specifically apply this method to excitatory-inhibitory networks with reciprocal motifs, and show that it yields reliable predictions for both the low-dimensional dynamics, and statistics of population activity. Importantly, it analytically accounts for the activity heterogeneity of individual neurons in specific realizations of local connectivity. Altogether, our approach allows us to disentangle the effects of mean connectivity and reciprocal motifs on the global recurrent feedback, and provides an intuitive picture of how local connectivity shapes global network dynamics.

Dynamic core-periphery structure of information sharing networks in entorhinal cortex and hippocampus

Neural computation is associated with the emergence, reconfiguration, and dissolution of cell assemblies in the context of varying oscillatory states. Here, we describe the complex spatiotemporal dynamics of cell assemblies through temporal network formalism. We use a sliding window approach to extract sequences of networks of information sharing among single units in hippocampus and entorhinal cortex during anesthesia and study how global and node-wise functional connectivity properties evolve through time and as a function of changing global brain state (theta vs. slow-wave oscillations). First, we find that information sharing networks display, at any time, a core-periphery structure in which an integrated core of more tightly functionally interconnected units links to more loosely connected network leaves. However the units participating to the core or to the periphery substantially change across time windows, with units entering and leaving the core in a smooth way. Second, we find that discrete network states can be defined on top of this continuously ongoing liquid core-periphery reorganization. Switching between network states results in a more abrupt modification of the units belonging to the core and is only loosely linked to transitions between global oscillatory states. Third, we characterize different styles of temporal connectivity that cells can exhibit within each state of the sharing network. While inhibitory cells tend to be central, we show that, otherwise, anatomical localization only poorly influences the patterns of temporal connectivity of the different cells. Furthermore, cells can change temporal connectivity style when the network changes state. Altogether, these findings reveal that the sharing of information mediated by the intrinsic dynamics of hippocampal and entorhinal cortex cell assemblies have a rich spatiotemporal structure, which could not have been identified by more conventional time- or state-averaged analyses of functional connectivity.

It is generally thought that computations performed by local brain circuits rely on complex neural processes, associated with the flexible waxing and waning of cell assemblies, that is, an ensemble of cells firing in tight synchrony. Although cell assembly formation is inherently and unavoidably dynamical, it is still common to find studies in which essentially “static” approaches are used to characterize this process. In the present study, we adopt instead a temporal network approach. Avoiding usual time-averaging procedures, we reveal that hub neurons are not hardwired but that cells vary smoothly their degree of integration within the assembly core. Furthermore, our temporal network framework enables the definition of alternative possible styles of “hubness.” Some cells may share information with a multitude of other units but only in an intermittent manner, as “activists” in a flash mob. In contrast, some other cells may share information in a steadier manner, as resolute “lobbyists.” Finally, by avoiding averages over preimposed states, we show that within each global oscillatory state rich switching dynamics can take place between a repertoire of many available network states. We thus show that the temporal network framework provides a natural and effective language to rigorously describe the rich spatiotemporal patterns of information sharing instantiated by cell assembly evolution.

Behavior of information flow near criticality

Recent experiments have indicated that many biological systems self-organize near their critical point, which hints at a common design principle. While it has been suggested that information transmission is optimized near the critical point, it remains unclear how information transmission depends on the dynamics of the input signal, the distance over which the information needs to be transmitted, and the distance to the critical point. Here we employ stochastic simulations of a driven two-dimensional Ising system and study the instantaneous mutual information and the information transmission rate between a driven input spin and an output spin. The instantaneous mutual information varies nonmonotonically with the temperature but increases monotonically with the correlation time of the input signal. In contrast, there exists not only an optimal temperature but also an optimal finite input correlation time that maximizes the information transmission rate. This global optimum arises from a fundamental trade-off between the need to maximize the frequency of independent input messages, the necessity to respond fast to changes in the input, and the need to respond reliably to these changes. The optimal temperature lies above the critical point but moves toward it as the distance between the input and output spin is increased.

Informational architecture across non-living and living collectives

Collective behavior is widely regarded as a hallmark property of living and intelligent systems. Yet, many examples are known of simple physical systems that are not alive, which nonetheless display collective behavior too, prompting simple physical models to often be adopted to explain living collective behaviors. To understand collective behavior as it occurs in living examples, it is important to determine whether or not there exist fundamental differences in how non-living and living systems act collectively, as well as the limits of the intuition that can be built from simpler, physical examples in explaining biological phenomenon. Here, we propose a framework for comparing non-living and living collectives as a continuum based on their information architecture: that is, how information is stored and processed across different degrees of freedom. We review diverse examples of collective phenomena, characterized from an information-theoretic perspective, and offer views on future directions for quantifying living collective behaviors based on their informational structure.

Review of tools and algorithms for network motif discovery in biological networks

Network motifs are recurrent and over-represented patterns having biological relevance. This is one of the important local properties of biological networks. Network motif discovery finds important applications in many areas such as functional analysis of biological components, the validity of network composition, classification of networks, disease discovery, identification of unique subunits etc. The discovery of network motifs is a computationally challenging task due to the large size of real networks, and the exponential increase of search space with respect to network size and motif size. This problem also includes the subgraph isomorphism check, which is Nondeterministic Polynomial (NP)-complete. Several tools and algorithms have been designed in the last few years to address this problem with encouraging results. These tools and algorithms can be classified into various categories based on exact census, mapping, pattern growth, and so on. In this study, critical aspects of network motif discovery, design principles of background algorithms, and their functionality have been reviewed with their strengths and limitations. The performances of state-of-art algorithms are discussed in terms of runtime efficiency, scalability, and space requirement. The future scope, research direction, and challenges of the existing algorithms are presented at the end of the study.

Deriving pairwise transfer entropy from network structure and motifs

Transfer entropy (TE) is an established method for quantifying directed statistical dependencies in neuroimaging and complex systems datasets. The pairwise (or bivariate) TE from a source to a target node in a network does not depend solely on the local source-target link weight, but on the wider network structure that the link is embedded in. This relationship is studied using a discrete-time linearly coupled Gaussian model, which allows us to derive the TE for each link from the network topology. It is shown analytically that the dependence on the directed link weight is only a first approximation, valid for weak coupling. More generally, the TE increases with the in-degree of the source and decreases with the in-degree of the target, indicating an asymmetry of information transfer between hubs and low-degree nodes. In addition, the TE is directly proportional to weighted motif counts involving common parents or multiple walks from the source to the target, which are more abundant in networks with a high clustering coefficient than in random networks. Our findings also apply to Granger causality, which is equivalent to TE for Gaussian variables. Moreover, similar empirical results on random Boolean networks suggest that the dependence of the TE on the in-degree extends to nonlinear dynamics.

Restricted information in a two-step cascade

A cell must sense extracellular and intracellular fluctuations and respond appropriately to survive for optimal cellular functioning. Accordingly, a cell builds up biochemical networks which can transduce information of extracellular and intracellular fluctuations accurately. We consider a generic two-step cascade as a model gene regulatory network containing three regulatory proteins S, X, and Y connected as S→X→Y. The intermediate node X is a stochastic variable, acts as an obstacle, and impedes the information flow from S to Y. We quantify the information that is restricted by X using the tools of information theory and term this as restricted information. In this context, we further propose two measurable quantities, restricted efficiency and information transfer efficiency. The former determines how efficiently X restricts the upstream information coming from S, while the latter computes the efficiency of X to pass the upstream information toward Y. We also quantify the information that is being uniquely transferred from X to Y, which determines the extent of the ability of X to act as a source of information. Our analysis shows that when the signal strength (or mean population of S, 〈s〉) is low, the intermediate X can carry forward the upstream information reliably as well, as it acts as a better source of information, thereby increasing the fidelity of the network. But at the high signal strength, X restricts most of the upstream information, and its ability to act as a source of information gets reduced. This leads to a loss of fidelity of the network.

Transitions in information processing dynamics at the whole-brain network level are driven by alterations in neural gain

A key component of the flexibility and complexity of the brain is its ability to dynamically adapt its functional network structure between integrated and segregated brain states depending on the demands of different cognitive tasks. Integrated states are prevalent when performing tasks of high complexity, such as maintaining items in working memory, consistent with models of a global workspace architecture. Recent work has suggested that the balance between integration and segregation is under the control of ascending neuromodulatory systems, such as the noradrenergic system, via changes in neural gain (in terms of the amplification and non-linearity in stimulus-response transfer function of brain regions). In a previous large-scale nonlinear oscillator model of neuronal network dynamics, we showed that manipulating neural gain parameters led to a 'critical' transition in phase synchrony that was associated with a shift from segregated to integrated topology, thus confirming our original prediction. In this study, we advance these results by demonstrating that the gain-mediated phase transition is characterized by a shift in the underlying dynamics of neural information processing. Specifically, the dynamics of the subcritical (segregated) regime are dominated by information storage, whereas the supercritical (integrated) regime is associated with increased information transfer (measured via transfer entropy). Operating near to the critical regime with respect to modulating neural gain parameters would thus appear to provide computational advantages, offering flexibility in the information processing that can be performed with only subtle changes in gain control. Our results thus link studies of whole-brain network topology and the ascending arousal system with information processing dynamics, and suggest that the constraints imposed by the ascending arousal system constrain low-dimensional modes of information processing within the brain.

Interaction Information Along Lifespan of the Resting Brain Dynamics Reveals a Major Redundant Role of the Default Mode Network

Interaction Information (II) generalizes the univariate Shannon entropy to triplets of variables, allowing the detection of redundant (R) or synergetic (S) interactions in dynamical networks. Here, we calculated II from functional magnetic resonance imaging data and asked whether R or S vary across brain regions and along lifespan. Preserved along lifespan, we found high overlapping between the pattern of high R and the default mode network, whereas high values of S were overlapping with different cognitive domains, such as spatial and temporal memory, emotion processing and motor skills. Moreover, we have found a robust balance between R and S among different age intervals, indicating informational compensatory mechanisms in brain networks.

Embedding the dynamics of a single delay system into a feed-forward ring

We investigate the relation between the dynamics of a single oscillator with delayed self-feedback and a feed-forward ring of such oscillators, where each unit is coupled to its next neighbor in the same way as in the self-feedback case. We show that periodic solutions of the delayed oscillator give rise to families of rotating waves with different wave numbers in the corresponding ring. In particular, if for the single oscillator the periodic solution is resonant to the delay, it can be embedded into a ring with instantaneous couplings. We discover several cases where the stability of a periodic solution for the single unit can be related to the stability of the corresponding rotating wave in the ring. As a specific example, we demonstrate how the complex bifurcation scenario of simultaneously emerging multijittering solutions can be transferred from a single oscillator with delayed pulse feedback to multijittering rotating waves in a sufficiently large ring of oscillators with instantaneous pulse coupling. Finally, we present an experimental realization of this dynamical phenomenon in a system of coupled electronic circuits of FitzHugh-Nagumo type.

The hidden simplicity of biology

Life is so remarkable, and so unlike any other physical system, that it is tempting to attribute special factors to it. Physics is founded on the assumption that universal laws and principles underlie all natural phenomena, but is it far from clear that there are 'laws of life' with serious descriptive or predictive power analogous to the laws of physics. Nor is there (yet) a 'theoretical biology' in the same sense as theoretical physics. Part of the obstacle in developing a universal theory of biological organization concerns the daunting complexity of living organisms. However, many attempts have been made to glimpse simplicity lurking within this complexity, and to capture this simplicity mathematically. In this paper we review a promising new line of inquiry to bring coherence and order to the realm of biology by focusing on 'information' as a unifying concept.

Are Nonlinear Model-Free Conditional Entropy Approaches for the Assessment of Cardiac Control Complexity Superior to the Linear Model-Based One?

OBJECTIVE

We test the hypothesis that the linear model-based (MB) approach for the estimation of conditional entropy (CE) can be utilized to assess the complexity of the cardiac control in healthy individuals.

METHODS

An MB estimate of CE was tested in an experimental protocol (i.e., the graded head-up tilt) known to produce a gradual decrease of cardiac control complexity as a result of the progressive vagal withdrawal and concomitant sympathetic activation. The MB approach was compared with traditionally exploited nonlinear model-free (MF) techniques such as corrected approximate entropy, sample entropy, corrected CE, two k -nearest-neighbor CE procedures and permutation CE. Electrocardiogram was recorded in 17 healthy subjects at rest in supine position and during head-up tilt with table angles of 15°, 30°, 45°, 60°, and 75°. Heart period (HP) was derived as the temporal distance between two consecutive R-wave peaks and analysis was carried out over stationary sequences of 256 successive HPs.

RESULTS

The performance of the MB method in following the progressive decrease of HP complexity with tilt table angles was in line with those of MF approaches and the MB index was remarkably correlated with the MF ones.

CONCLUSION

The MB approach can be utilized to monitor the changes of the complexity of the cardiac control, thus speeding up dramatically the CE calculation.

SIGNIFICANCE

The remarkable performance of the MB approach challenges the notion, generally assumed in cardiac control complexity analysis based on CE, about the need of MF techniques and could allow real-time applications.

The informational architecture of the cell

We compare the informational architecture of biological and random networks to identify informational features that may distinguish biological networks from random. The study presented here focuses on the Boolean network model for regulation of the cell cycle of the fission yeast Schizosaccharomyces pombe. We compare calculated values of local and global information measures for the fission yeast cell cycle to the same measures as applied to two different classes of random networks: Erdös-Rényi and scale-free. We report patterns in local information processing and storage that do indeed distinguish biological from random, associated with control nodes that regulate the function of the fission yeast cell-cycle network. Conversely, we find that integrated information, which serves as a global measure of 'emergent' information processing, does not differ from random for the case presented. We discuss implications for our understanding of the informational architecture of the fission yeast cell-cycle network in particular, and more generally for illuminating any distinctive physics that may be operative in life.

Disentangling cardiovascular control mechanisms during head-down tilt via joint transfer entropy and self-entropy decompositions

A full decomposition of the predictive entropy (PE) of the spontaneous variations of the heart period (HP) given systolic arterial pressure (SAP) and respiration (R) is proposed. The PE of HP is decomposed into the joint transfer entropy (JTE) from SAP and R to HP and self-entropy (SE) of HP. The SE is the sum of three terms quantifying the synergistic/redundant contributions of HP and SAP, when taken individually and jointly, to SE and one term conditioned on HP and SAP denoted as the conditional SE (CSE) of HP given SAP and R. The JTE from SAP and R to HP is the sum of two terms attributable to SAP or R plus an extra term describing the redundant/synergistic contribution to the JTE. All quantities were computed during cardiopulmonary loading induced by −25° head-down tilt (HDT) via a multivariate linear regression approach. We found that: (i) the PE of HP decreases during HDT; (ii) the decrease of PE is attributable to a lessening of SE of HP, while the JTE from SAP and R to HP remains constant; (iii) the SE of HP is dominant over the JTE from SAP and R to HP and the CSE of HP given SAP and R is prevailing over the SE of HP due to SAP and R both in supine position and during HDT; (iv) all terms of the decompositions of JTE from SAP and R to HP and SE of HP due to SAP and R were not affected by HDT; (v) the decrease of the SE of HP during HDT was attributed to the reduction of the CSE of HP given SAP and R; (vi) redundancy of SAP and R is prevailing over synergy in the information transferred into HP both in supine position and during HDT, while in the HP information storage synergy and redundancy are more balanced. The approach suggests that the larger complexity of the cardiac control during HDT is unrelated to the baroreflex control and cardiopulmonary reflexes and may be related to central commands and/or modifications of the dynamical properties of the sinus node.

Conditional Self-Entropy and Conditional Joint Transfer Entropy in Heart Period Variability during Graded Postural Challenge

Self-entropy (SE) and transfer entropy (TE) are widely utilized in biomedical signal processing to assess the information stored into a system and transferred from a source to a destination respectively. The study proposes a more specific definition of the SE, namely the conditional SE (CSE), and a more flexible definition of the TE based on joint TE (JTE), namely the conditional JTE (CJTE), for the analysis of information dynamics in multivariate time series. In a protocol evoking a gradual sympathetic activation and vagal withdrawal proportional to the magnitude of the orthostatic stimulus, such as the graded head-up tilt, we extracted the beat-to-beat spontaneous variability of heart period (HP), systolic arterial pressure (SAP) and respiratory activity (R) in 19 healthy subjects and we computed SE of HP, CSE of HP given SAP and R, JTE from SAP and R to HP, CJTE from SAP and R to HP given SAP and CJTE from SAP and R to HP given R. CSE of HP given SAP and R was significantly smaller than SE of HP and increased progressively with the amplitude of the stimulus, thus suggesting that dynamics internal to HP and unrelated to SAP and R, possibly linked to sympathetic activation evoked by head-up tilt, might play a role during the orthostatic challenge. While JTE from SAP and R to HP was independent of tilt table angle, CJTE from SAP and R to HP given R and from SAP and R to HP given SAP showed opposite trends with tilt table inclination, thus suggesting that the importance of the cardiac baroreflex increases and the relevance of the cardiopulmonary pathway decreases during head-up tilt. The study demonstrates the high specificity of CSE and the high flexibility of CJTE over real data and proves that they are particularly helpful in disentangling physiological mechanisms and in assessing their different contributions to the overall cardiovascular regulation.

Local active information storage as a tool to understand distributed neural information processing

Every act of information processing can in principle be decomposed into the component operations of information storage, transfer, and modification. Yet, while this is easily done for today's digital computers, the application of these concepts to neural information processing was hampered by the lack of proper mathematical definitions of these operations on information. Recently, definitions were given for the dynamics of these information processing operations on a local scale in space and time in a distributed system, and the specific concept of local active information storage was successfully applied to the analysis and optimization of artificial neural systems. However, no attempt to measure the space-time dynamics of local active information storage in neural data has been made to date. Here we measure local active information storage on a local scale in time and space in voltage sensitive dye imaging data from area 18 of the cat. We show that storage reflects neural properties such as stimulus preferences and surprise upon unexpected stimulus change, and in area 18 reflects the abstract concept of an ongoing stimulus despite the locally random nature of this stimulus. We suggest that LAIS will be a useful quantity to test theories of cortical function, such as predictive coding.

Identification of breast cancer patients based on human signaling network motifs

Identifying breast cancer patients is crucial to the clinical diagnosis and therapy for this disease. Conventional gene-based methods for breast cancer diagnosis ignore gene-gene interactions and thus may lead to loss of power. In this study, we proposed a novel method to select classification features, called “Selection of Significant Expression-Correlation Differential Motifs” (SSECDM). This method applied a network motif-based approach, combining a human signaling network and high-throughput gene expression data to distinguish breast cancer samples from normal samples. Our method has higher classification performance and better classification accuracy stability than the mutual information (MI) method or the individual gene sets method. It may become a useful tool for identifying and treating patients with breast cancer and other cancers, thus contributing to clinical diagnosis and therapy for these diseases.