Emergence of informative higher scales in biological systems: a computational toolkit for optimal prediction and control

A decade of thermostatted kinetic theory models for complex active matter living systems

In the last decade, the thermostatted kinetic theory has been proposed as a general paradigm for the modeling of complex systems of the active matter and, in particular, in biology. Homogeneous and inhomogeneous frameworks of the thermostatted kinetic theory have been employed for modeling phenomena that are the result of interactions among the elements, called active particles, composing the system. Functional subsystems contain heterogeneous active particles that are able to perform the same task, called activity. Active matter living systems usually operate out-of-equilibrium; accordingly, a mathematical thermostat is introduced in order to regulate the fluctuations of the activity of particles. The time evolution of the functional subsystems is obtained by introducing the conservative and the nonconservative interactions which represent activity-transition, natural birth/death, induced proliferation/destruction, and mutation of the active particles. This review paper is divided in two parts: In the first part the review deals with the mathematical frameworks of the thermostatted kinetic theory that can be found in the literature of the last decade and a unified approach is proposed; the second part of the review is devoted to the specific mathematical models derived within the thermostatted kinetic theory presented in the last decade for complex biological systems, such as wound healing diseases, the recognition process and the learning dynamics of the human immune system, the hiding-learning dynamics and the immunoediting process occurring during the cancer-immune system competition. Future research perspectives are discussed from the theoretical and application viewpoints, which suggest the important interplay among the different scholars of the applied sciences and the desire of a multidisciplinary approach or rather a theory for the modeling of every active matter system.

Collective intelligence: A unifying concept for integrating biology across scales and substrates

A defining feature of biology is the use of a multiscale architecture, ranging from molecular networks to cells, tissues, organs, whole bodies, and swarms. Crucially however, biology is not only nested structurally, but also functionally: each level is able to solve problems in distinct problem spaces, such as physiological, morphological, and behavioral state space. Percolating adaptive functionality from one level of competent subunits to a higher functional level of organization requires collective dynamics: multiple components must work together to achieve specific outcomes. Here we overview a number of biological examples at different scales which highlight the ability of cellular material to make decisions that implement cooperation toward specific homeodynamic endpoints, and implement collective intelligence by solving problems at the cell, tissue, and whole-organism levels. We explore the hypothesis that collective intelligence is not only the province of groups of animals, and that an important symmetry exists between the behavioral science of swarms and the competencies of cells and other biological systems at different scales. We then briefly outline the implications of this approach, and the possible impact of tools from the field of diverse intelligence for regenerative medicine and synthetic bioengineering.

Emergence and Causality in Complex Systems: A Survey of Causal Emergence and Related Quantitative Studies

Emergence and causality are two fundamental concepts for understanding complex systems. They are interconnected. On one hand, emergence refers to the phenomenon where macroscopic properties cannot be solely attributed to the cause of individual properties. On the other hand, causality can exhibit emergence, meaning that new causal laws may arise as we increase the level of abstraction. Causal emergence (CE) theory aims to bridge these two concepts and even employs measures of causality to quantify emergence. This paper provides a comprehensive review of recent advancements in quantitative theories and applications of CE. It focuses on two primary challenges: quantifying CE and identifying it from data. The latter task requires the integration of machine learning and neural network techniques, establishing a significant link between causal emergence and machine learning. We highlight two problem categories: CE with machine learning and CE for machine learning, both of which emphasize the crucial role of effective information (EI) as a measure of causal emergence. The final section of this review explores potential applications and provides insights into future perspectives.

On the Nature of Information: How FAIR Digital Objects are Building-up Semantic Space

In this paper, we are concerned about the nature of information and how to gather and compose data with the help of so called FAIR digital objects (FDOs) in order to transform them to knowledge. FDOs are digital surrogates of real objects. The nature of information is intrinsically linked to the kind of questions one is asking. One might not ask a question or get philosophical about it. Answers depend on the data different disciplines gather about their objects of study. In Statistical Physics, classical Shannon entropy measures system order which in equilibrium just equals the heat exchanged with the environment. In cell biology, each protein carries certain functions which create specific information. Cognitive science describes how organisms perceive their environment via functional sensors and control behavior accordingly. Note that one can have function and control without meaning. In contrast, psychology is concerned with the assessment of our perceptions by assigning meaning and ensuing actions. Finally, philosophy builds logical constructs and formulates principles, in effect transforming facts into complex knowledge. All these statements make sense, but there is an even more concise way. Indeed, Luciano Floridi provides a precise and thorough classification of information in his central oeuvre On the Philosophy of Information (Floridi 2013). Especially, he performs a sequential construction to develop the attributes which data need to have in order to count as knowledge. Semantic information is necessarily well-formed, meaningful and truthful. Well-formed data becomes meaningful by action based-semantics of an autonomous-agent solving the symbol grounding problem (Taddeo and Floridi 2005) interacting with the environment. Knowledge is created then by being informed through relevant data accounted for. We notice that the notion of agency is crucial for defining meaning. The apparent gap between Sciences and Humanities (Bawden and Robinson 2020) is created by the very existence of meaning. Further, meaning depends on interactions & connotations which are commensurate with the effective complexity of the environment of a particular agent resulting in an array of possible definitions.

In his classical paper More is different (Anderson 1972) discussed verbatim the hierarchical nature of science. Each level is made of and obeys the laws of its constituents from one level below with the higher-level exhibiting emergent properties like wetness of water assignable only to the whole system. As we rise through the hierarchies, there is a branch of science for each level of complexity; on each complexity level there are objects for which it is appropriate and fitting to build up vocabulary for the respective levels of description leading to formation of disciplinary languages. It is the central idea of causal emergence that on each level there is an optimal degree of coarse graining to define those objects in such a way that causality becomes maximal between them. This means there is emergence of informative higher scales in complex materials extending to biological systems and into the brain with its neural networks representing our thoughts in a hierarchy of neural correlates. A computational toolkit for optimal level prediction and control has been developed (Hoel and Levin 2020) which was conceptually extended to integrated information theory of consciousness (Albantakis et al. 2019). The large gap between sciences and humanities discussed above exhibits itself in a series of small gaps connected to the emergence of informative higher scales. It has been suggested that the origin of life may be identified as a transition in causal structure and information flow (Walker 2014). Integrated information measures globally how much the causal mechanisms of a system reduce the uncertainty about the possible causes for a given state. A measure of “information flow” that accurately captures causal effects has been proposed (Ay and Polani 2008). The state of the art is presented in (Ay et al. 2022) where the link between information and complexity is discussed. Ay et al single out hierarchical systems and interlevel causation. Even further, (Rosas et al. 2020) reconcile conflicting views of emergence via an exact information-theoretic approach to identify causal emergence in multivariate data. As information becomes differentially richer one eventually needs complexity measures beyond {Rn}. One may define generalized metrices on these spaces (Pirr&oacute 2009) measuring information complexity on ever higher hierarchical levels of information. As one rises through hierarchies, information on higher scale is usually gained by coarse graining to arrive at an effective, nevertheless exact description, on the higher scale. It is repeated coarse graining of syntactically well-ordered information layers which eventually leads to semantic information in a process which I conjecture to be reminiscent of renormalization group flow leading to a universal classification scheme. Thus, we identify scientific disciplines and their corresponding data sets as dual universality classes of physical and epistemic structure formation, respectively. Above the semantic gap, we may call this process quantification of the qualitative by semantic metrics. Indeed, (Kolchinsky and Wolpert 2018) explored for the first time quantitative semantic concepts in Physics in their 2018 seminal paper entitled Semantic information, autonomous agency and non-equilibrium statistical physics. Their measures are numeric variants of entropy. Semantic information is identified with ‘the information that a physical system has about its environment that is causally necessary for the system to maintain its own existence over time’.

FDOs are employed in these processes in two fundamental ways. For practical implementations of FDO technology, see accompanying abstract (Wittenburg et al. 2022). First, the FAIR principles (Wilkinson et al. 2016) ensure that unconnected pieces of data may be percolated into an integrated data space. Percolation creates the information density needed to feed AI-driven built up of semantic space. Without FDOs we wouldn't have the gravity for this to occur. Second, the very structure of FDOs, capable of symmetry preserving or breaking fusion events into composed entities, makes them homologous to mathematical categories. This will proof to be a powerful tool to unravel the nature of information via analyzing its topological structure algebraically, especially when considering our conjecture concerning universality, classes of information and their possible instantiations on vastly different length and time scales, in effect explaining analogous structure formation.

Emergence as the conversion of information: a unifying theory

Is reduction always a good scientific strategy? The existence of the special sciences above physics suggests not. Previous research has shown that dimensionality reduction (macroscales) can increase the dependency between elements of a system (a phenomenon called 'causal emergence'). Here, we provide an umbrella mathematical framework for emergence based on information conversion. We show evidence that coarse-graining can convert information from one 'type' to another. We demonstrate this using the well-understood mutual information measure applied to Boolean networks. Using partial information decomposition, the mutual information can be decomposed into redundant, unique and synergistic information atoms. Then by introducing a novel measure of the synergy bias of a given decomposition, we are able to show that the synergy component of a Boolean network's mutual information can increase at macroscales. This can occur even when there is no difference in the total mutual information between a macroscale and its underlying microscale, proving information conversion. We relate this broad framework to previous work, compare it to other theories, and argue it complexifies any notion of universal reduction in the sciences, since such reduction would likely lead to a loss of synergistic information in scientific models. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'.

Degeneracy measures in biologically plausible random Boolean networks

Background

Degeneracy—the ability of structurally different elements to perform similar functions—is a property of many biological systems. Highly degenerate systems show resilience to perturbations and damage because the system can compensate for compromised function due to reconfiguration of the underlying network dynamics. Degeneracy thus suggests how biological systems can thrive despite changes to internal and external demands. Although degeneracy is a feature of network topologies and seems to be implicated in a wide variety of biological processes, research on degeneracy in biological networks is mostly limited to weighted networks. In this study, we test an information theoretic definition of degeneracy on random Boolean networks, frequently used to model gene regulatory networks. Random Boolean networks are discrete dynamical systems with binary connectivity and thus, these networks are well-suited for tracing information flow and the causal effects. By generating networks with random binary wiring diagrams, we test the effects of systematic lesioning of connections and perturbations of the network nodes on the degeneracy measure.

Results

Our analysis shows that degeneracy, on average, is the highest in networks in which ~ 20% of the connections are lesioned while 50% of the nodes are perturbed. Moreover, our results for the networks with no lesions and the fully-lesioned networks are comparable to the degeneracy measures from weighted networks, thus we show that the degeneracy measure is applicable to different networks.

Conclusions

Such a generalized applicability implies that degeneracy measures may be a useful tool for investigating a wide range of biological networks and, therefore, can be used to make predictions about the variety of systems’ ability to recover function.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04601-5.

Revealing semantic and emotional structure of suicide notes with cognitive network science

Understanding how people who commit suicide perceive their cognitive states and emotions represents an important open scientific challenge. We build upon cognitive network science, psycholinguistics and semantic frame theory to introduce a network representation of suicidal ideation as expressed in multiple suicide notes. By reconstructing the knowledge structure of such notes, we reveal interconnections between the ideas and emotional states of people who committed suicide through an analysis of emotional balance motivated by structural balance theory, semantic prominence and emotional profiling. Our results indicate that connections between positively- and negatively-valenced terms give rise to a degree of balance that is significantly higher than in a null model where the affective structure is randomized and in a linguistic baseline model capturing mind-wandering in absence of suicidal ideation. We show that suicide notes are affectively compartmentalized such that positive concepts tend to cluster together and dominate the overall network structure. Notably, this positive clustering diverges from perceptions of self, which are found to be dominated by negative, sad conceptual associations in analyses based on subject-verb-object relationships and emotional profiling. A key positive concept is "love", which integrates information relating the self to others and is semantically prominent across suicide notes. The emotions constituting the semantic frame of "love" combine joy and trust with anticipation and sadness, which can be linked to psychological theories of meaning-making as well as narrative psychology. Our results open new ways for understanding the structure of genuine suicide notes and may be used to inform future research on suicide prevention.

On the prediction of neuronal microscale topology descriptors based on mesoscale recordings

The brain possesses structural and functional hierarchical architectures organized over multiple scales. Considering that functional recordings commonly focused on a single spatial level, and because multiple scales interact with one another, we explored the behaviour of in silico neuronal networks across different scales. We established ad hoc relations of several topological descriptors (average clustering coefficient, average path length, small-world propensity, modularity, network degree, synchronizability and fraction of long-term connections) between different scales upon application and empirical validation of a Euclidian renormalization approach. We tested a simple network (distance-dependent model) as well as an artificial cortical network (Vertex; undirected and directed networks) finding the same qualitative power law relations of the parameters across levels: their quantitative nature is model dependent. Those findings were then organized in a workflow that can be used to predict, with approximation, microscale topologies from mesoscale recordings. The present manuscript not only presents a theoretical framework for the renormalization of biological neuronal network and their study across scales in light of the spatial features of the recording method but also proposes an applicable workflow to compare real functional networks across scales.

Message Passing and Metabolism

Active inference is an increasingly prominent paradigm in theoretical biology. It frames the dynamics of living systems as if they were solving an inference problem. This rests upon their flow towards some (non-equilibrium) steady state-or equivalently, their maximisation of the Bayesian model evidence for an implicit probabilistic model. For many models, these self-evidencing dynamics manifest as messages passed among elements of a system. Such messages resemble synaptic communication at a neuronal network level but could also apply to other network structures. This paper attempts to apply the same formulation to biochemical networks. The chemical computation that occurs in regulation of metabolism relies upon sparse interactions between coupled reactions, where enzymes induce conditional dependencies between reactants. We will see that these reactions may be viewed as the movement of probability mass between alternative categorical states. When framed in this way, the master equations describing such systems can be reformulated in terms of their steady-state distribution. This distribution plays the role of a generative model, affording an inferential interpretation of the underlying biochemistry. Finally, we see that-in analogy with computational neurology and psychiatry-metabolic disorders may be characterized as false inference under aberrant prior beliefs.

Causal Geometry

Information geometry has offered a way to formally study the efficacy of scientific models by quantifying the impact of model parameters on the predicted effects. However, there has been little formal investigation of causation in this framework, despite causal models being a fundamental part of science and explanation. Here, we introduce causal geometry, which formalizes not only how outcomes are impacted by parameters, but also how the parameters of a model can be intervened upon. Therefore, we introduce a geometric version of "effective information"-a known measure of the informativeness of a causal relationship. We show that it is given by the matching between the space of effects and the space of interventions, in the form of their geometric congruence. Therefore, given a fixed intervention capability, an effective causal model is one that is well matched to those interventions. This is a consequence of "causal emergence," wherein macroscopic causal relationships may carry more information than "fundamental" microscopic ones. We thus argue that a coarse-grained model may, paradoxically, be more informative than the microscopic one, especially when it better matches the scale of accessible interventions-as we illustrate on toy examples.

Examining the Causal Structures of Deep Neural Networks Using Information Theory

Deep Neural Networks (DNNs) are often examined at the level of their response to input, such as analyzing the mutual information between nodes and data sets. Yet DNNs can also be examined at the level of causation, exploring "what does what" within the layers of the network itself. Historically, analyzing the causal structure of DNNs has received less attention than understanding their responses to input. Yet definitionally, generalizability must be a function of a DNN's causal structure as it reflects how the DNN responds to unseen or even not-yet-defined future inputs. Here, we introduce a suite of metrics based on information theory to quantify and track changes in the causal structure of DNNs during training. Specifically, we introduce the effective information (EI) of a feedforward DNN, which is the mutual information between layer input and output following a maximum-entropy perturbation. The EI can be used to assess the degree of causal influence nodes and edges have over their downstream targets in each layer. We show that the EI can be further decomposed in order to examine the sensitivity of a layer (measured by how well edges transmit perturbations) and the degeneracy of a layer (measured by how edge overlap interferes with transmission), along with estimates of the amount of integrated information of a layer. Together, these properties define where each layer lies in the "causal plane", which can be used to visualize how layer connectivity becomes more sensitive or degenerate over time, and how integration changes during training, revealing how the layer-by-layer causal structure differentiates. These results may help in understanding the generalization capabilities of DNNs and provide foundational tools for making DNNs both more generalizable and more explainable.

Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data

The broad concept of emergence is instrumental in various of the most challenging open scientific questions-yet, few quantitative theories of what constitutes emergent phenomena have been proposed. This article introduces a formal theory of causal emergence in multivariate systems, which studies the relationship between the dynamics of parts of a system and macroscopic features of interest. Our theory provides a quantitative definition of downward causation, and introduces a complementary modality of emergent behaviour-which we refer to as causal decoupling. Moreover, the theory allows practical criteria that can be efficiently calculated in large systems, making our framework applicable in a range of scenarios of practical interest. We illustrate our findings in a number of case studies, including Conway's Game of Life, Reynolds' flocking model, and neural activity as measured by electrocorticography.

Emergence of informative higher scales in biological systems: a computational toolkit for optimal prediction and control

The biological sciences span many spatial and temporal scales in attempts to understand the function and evolution of complex systems-level processes, such as embryogenesis. It is generally assumed that the most effective description of these processes is in terms of molecular interactions. However, recent developments in information theory and causal analysis now allow for the quantitative resolution of this question. In some cases, macro-scale models can minimize noise and increase the amount of information an experimenter or modeler has about "what does what." This result has numerous implications for evolution, pattern regulation, and biomedical strategies. Here, we provide an introduction to these quantitative techniques, and use them to show how informative macro-scales are common across biology. Our goal is to give biologists the tools to identify the maximally-informative scale at which to model, experiment on, predict, control, and understand complex biological systems.