Information theory, predictability and the emergence of complex life

Fundamental constraints to the logic of living systems

It has been argued that the historical nature of evolution makes it a highly path-dependent process. Under this view, the outcome of evolutionary dynamics could have resulted in organisms with different forms and functions. At the same time, there is ample evidence that convergence and constraints strongly limit the domain of the potential design principles that evolution can achieve. Are these limitations relevant in shaping the fabric of the possible? Here, we argue that fundamental constraints are associated with the logic of living matter. We illustrate this idea by considering the thermodynamic properties of living systems, the linear nature of molecular information, the cellular nature of the building blocks of life, multicellularity and development, the threshold nature of computations in cognitive systems and the discrete nature of the architecture of ecosystems. In all these examples, we present available evidence and suggest potential avenues towards a well-defined theoretical formulation.

How Turing parasites expand the computational landscape of digital life

Why are living systems complex? Why does the biosphere contain living beings with complexity features beyond those of the simplest replicators? What kind of evolutionary pressures result in more complex life forms? These are key questions that pervade the problem of how complexity arises in evolution. One particular way of tackling this is grounded in an algorithmic description of life: living organisms can be seen as systems that extract and process information from their surroundings to reduce uncertainty. Here we take this computational approach using a simple bit string model of coevolving agents and their parasites. While agents try to predict their worlds, parasites do the same with their hosts. The result of this process is that, to escape their parasites, the host agents expand their computational complexity despite the cost of maintaining it. This, in turn, is followed by increasingly complex parasitic counterparts. Such arms races display several qualitative phases, from monotonous to punctuated evolution or even ecological collapse. Our minimal model illustrates the relevance of parasites in providing an active mechanism for expanding living complexity beyond simple replicators, suggesting that parasitic agents are likely to be a major evolutionary driver for biological complexity.

Plausible pathway for a host-parasite molecular replication network to increase its complexity through Darwinian evolution

How the complexity of primitive self-replication molecules develops through Darwinian evolution remains a mystery with regards to the origin of life. Theoretical studies have proposed that coevolution with parasitic replicators increases network complexity by inducing inter-dependent replication. Particularly, Takeuchi and Hogeweg proposed a complexification process of replicator networks by successive appearance of a parasitic replicator followed by the addition of a new host replicator that is resistant to the parasitic replicator. However, the feasibility of such complexification with biologically relevant molecules is still unknown owing to the lack of an experimental model. Here, we investigated the plausible complexification pathway of host-parasite replicators using both an experimental host-parasite RNA replication system and a theoretical model based on the experimental system. We first analyzed the parameter space that allows for sustainable replication in various replication networks ranging from a single molecule to three-member networks using computer simulation. The analysis shows that the most plausible complexification pathway from a single host replicator is the addition of a parasitic replicator, followed by the addition of a new host replicator that is resistant to the parasite, consistent with the previous study by Takeuchi and Hogeweg. We also provide evidence that the pathway actually occurred in our previous evolutionary experiment. These results provide experimental evidence that a population of a single replicator spontaneously evolves into multi-replicator networks through coevolution with parasitic replicators.

From the origin of life to pandemics: emergent phenomena in complex systems

When a large number of similar entities interact among each other and with their environment at a low scale, unexpected outcomes at higher spatio-temporal scales might spontaneously arise. This non-trivial phenomenon, known as emergence, characterizes a broad range of distinct complex systems-from physical to biological and social-and is often related to collective behaviour. It is ubiquitous, from non-living entities such as oscillators that under specific conditions synchronize, to living ones, such as birds flocking or fish schooling. Despite the ample phenomenological evidence of the existence of systems' emergent properties, central theoretical questions to the study of emergence remain unanswered, such as the lack of a widely accepted, rigorous definition of the phenomenon or the identification of the essential physical conditions that favour emergence. We offer here a general overview of the phenomenon of emergence and sketch current and future challenges on the topic. Our short review also serves as an introduction to the theme issue Emergent phenomena in complex physical and socio-technical systems: from cells to societies, where we provide a synthesis of the contents tackled in the issue and outline how they relate to these challenges, spanning from current advances in our understanding on the origin of life to the large-scale propagation of infectious diseases. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'.

Evolution of Brains and Computers: The Roads Not Taken

When computers started to become a dominant part of technology around the 1950s, fundamental questions about reliable designs and robustness were of great relevance. Their development gave rise to the exploration of new questions, such as what made brains reliable (since neurons can die) and how computers could get inspiration from neural systems. In parallel, the first artificial neural networks came to life. Since then, the comparative view between brains and computers has been developed in new, sometimes unexpected directions. With the rise of deep learning and the development of connectomics, an evolutionary look at how both hardware and neural complexity have evolved or designed is required. In this paper, we argue that important similarities have resulted both from convergent evolution (the inevitable outcome of architectural constraints) and inspiration of hardware and software principles guided by toy pictures of neurobiology. Moreover, dissimilarities and gaps originate from the lack of major innovations that have paved the way to biological computing (including brains) that are completely absent within the artificial domain. As it occurs within synthetic biocomputation, we can also ask whether alternative minds can emerge from A.I. designs. Here, we take an evolutionary view of the problem and discuss the remarkable convergences between living and artificial designs and what are the pre-conditions to achieve artificial intelligence.

A Mutation Threshold for Cooperative Takeover

One of the leading theories for the origin of life includes the hypothesis according to which life would have evolved as cooperative networks of molecules. Explaining cooperation—and particularly, its emergence in favoring the evolution of life-bearing molecules—is thus a key element in describing the transition from nonlife to life. Using agent-based modeling of the iterated prisoner’s dilemma, we investigate the emergence of cooperative behavior in a stochastic and spatially extended setting and characterize the effects of inheritance and variability. We demonstrate that there is a mutation threshold above which cooperation is—counterintuitively—selected, which drives a dramatic and robust cooperative takeover of the whole system sustained consistently up to the error catastrophe, in a manner reminiscent of typical phase transition phenomena in statistical physics. Moreover, our results also imply that one of the simplest conditional cooperative strategies, “Tit-for-Tat”, plays a key role in the emergence of cooperative behavior required for the origin of life.

Towards the Idea of Molecular Brains

How can single cells without nervous systems perform complex behaviours such as habituation, associative learning and decision making, which are considered the hallmark of animals with a brain? Are there molecular systems that underlie cognitive properties equivalent to those of the brain? This review follows the development of the idea of molecular brains from Darwin’s “root brain hypothesis”, through bacterial chemotaxis, to the recent discovery of neuron-like r-protein networks in the ribosome. By combining a structural biology view with a Bayesian brain approach, this review explores the evolutionary labyrinth of information processing systems across scales. Ribosomal protein networks open a window into what were probably the earliest signalling systems to emerge before the radiation of the three kingdoms. While ribosomal networks are characterised by long-lasting interactions between their protein nodes, cell signalling networks are essentially based on transient interactions. As a corollary, while signals propagated in persistent networks may be ephemeral, networks whose interactions are transient constrain signals diffusing into the cytoplasm to be durable in time, such as post-translational modifications of proteins or second messenger synthesis. The duration and nature of the signals, in turn, implies different mechanisms for the integration of multiple signals and decision making. Evolution then reinvented networks with persistent interactions with the development of nervous systems in metazoans. Ribosomal protein networks and simple nervous systems display architectural and functional analogies whose comparison could suggest scale invariance in information processing. At the molecular level, the significant complexification of eukaryotic ribosomal protein networks is associated with a burst in the acquisition of new conserved aromatic amino acids. Knowing that aromatic residues play a critical role in allosteric receptors and channels, this observation suggests a general role of π systems and their interactions with charged amino acids in multiple signal integration and information processing. We think that these findings may provide the molecular basis for designing future computers with organic processors.

Metacode: One code to rule them all

The code of codes or metacode is a microcosm where biological layers, as well as their codes, interact together allowing the continuity of information flow in organisms by increasing biological entities' complexity. Through this novel organic code, biological systems scale towards niches with higher informatic freedom building structures that increase the entropy in the universe. Code biology has developed a novel informational framework where biological entities strive themselves through the information flow carried out through organic codes consisting of two molecular or functional landscapes intertwined through arbitrary linkages via an adaptor whose nature is autonomous from molecular determinism. Here we will integrate genomic and epigenomic codes according to the evidence released in ENCODE (phase 3), psychENCODE and GTEx project, outlining the principles of the metacode, to address the continuous nature of biological systems and their inter-layered information flow. This novel complex metacode maps from very constrained sets of elements (i.e., regulation sites modulating gene expression) to new ones with greater freedom of decoding (i.e., a continuous cell phenotypic space). This leads to a new domain in code biology where biological systems are informatic attractors that navigate an energy metaspace through a complexity-noise balance, stalling in emergent niches where organic codes take meaning.

Bounding generalized relative entropies: Nonasymptotic quantum speed limits

Information theory has become an increasingly important research field to better understand quantum mechanics. Noteworthy, it covers both foundational and applied perspectives, also offering a common technical language to study a variety of research areas. Remarkably, one of the key information-theoretic quantities is given by the relative entropy, which quantifies how difficult is to tell apart two probability distributions, or even two quantum states. Such a quantity rests at the core of fields like metrology, quantum thermodynamics, quantum communication, and quantum information. Given this broadness of applications, it is desirable to understand how this quantity changes under a quantum process. By considering a general unitary channel, we establish a bound on the generalized relative entropies (Rényi and Tsallis) between the output and the input of the channel. As an application of our bounds, we derive a family of quantum speed limits based on relative entropies. Possible connections between this family with thermodynamics, quantum coherence, asymmetry, and single-shot information theory are briefly discussed.

Fate of Duplicated Neural Structures

Statistical physics determines the abundance of different arrangements of matter depending on cost-benefit balances. Its formalism and phenomenology percolate throughout biological processes and set limits to effective computation. Under specific conditions, self-replicating and computationally complex patterns become favored, yielding life, cognition, and Darwinian evolution. Neurons and neural circuits sit at a crossroads between statistical physics, computation, and (through their role in cognition) natural selection. Can we establish a statistical physics of neural circuits? Such theory would tell what kinds of brains to expect under set energetic, evolutionary, and computational conditions. With this big picture in mind, we focus on the fate of duplicated neural circuits. We look at examples from central nervous systems, with stress on computational thresholds that might prompt this redundancy. We also study a naive cost-benefit balance for duplicated circuits implementing complex phenotypes. From this, we derive phase diagrams and (phase-like) transitions between single and duplicated circuits, which constrain evolutionary paths to complex cognition. Back to the big picture, similar phase diagrams and transitions might constrain I/O and internal connectivity patterns of neural circuits at large. The formalism of statistical physics seems to be a natural framework for this worthy line of research.

Decoding collective communications using information theory tools

Organisms have evolved sensory mechanisms to extract pertinent information from their environment, enabling them to assess their situation and act accordingly. For social organisms travelling in groups, like the fish in a school or the birds in a flock, sharing information can further improve their situational awareness and reaction times. Data on the benefits and costs of social coordination, however, have largely allowed our understanding of why collective behaviours have evolved to outpace our mechanistic knowledge of how they arise. Recent studies have begun to correct this imbalance through fine-scale analyses of group movement data. One approach that has received renewed attention is the use of information theoretic (IT) tools like mutual information, transfer entropy and causation entropy, which can help identify causal interactions in the type of complex, dynamical patterns often on display when organisms act collectively. Yet, there is a communications gap between studies focused on the ecological constraints and solutions of collective action with those demonstrating the promise of IT tools in this arena. We attempt to bridge this divide through a series of ecologically motivated examples designed to illustrate the benefits and challenges of using IT tools to extract deeper insights into the interaction patterns governing group-level dynamics. We summarize some of the approaches taken thus far to circumvent existing challenges in this area and we conclude with an optimistic, yet cautionary perspective.

Ion-Based Cellular Signal Transmission, Principles of Minimum Information Loss, and Evolution by Natural Selection

The Extreme Physical Information EPI principle states that maximum information transmission or, equivalently, a minimum information loss is a fundamental property of nature. Prior work has demonstrated the universal EPI principle allows derivation of nearly all physical laws. Here, we investigate whether EPI can similarly give rise to the fundamental law of life: Evolution. Living systems require information to survive and proliferate. Heritable information in the genome encodes the structure and function of cellular macromolecules but this information remains fixed over time. In contrast, a cell must rapidly and continuously access, analyze, and respond to a wide range of continuously changing spatial and temporal information in the environment. We propose these two information dynamics are linked because the genes encode the structure of the macromolecules that form information conduits necessary for the dynamical interactions with the external environment. However, because the genome does not have the capacity to precisely locate the time and location of external signals, we propose the cell membrane is the site at which most external information is received and processed. In our model, an external signal is detected by gates on transmembrane ion channel and transmitted into the cytoplasm through ions that flow along pre-existing concentration gradients when the gate opens. The resulting cytoplasmic ion “puff” is localized in both time and space, thus producing spatial and temporal information. Small, localized signals in the cytoplasm are “processed” through alterations in the function and location of peripheral membrane proteins. Larger perturbations produce prolonged or spatially extensive changes in cytoplasmic ion concentrations that can be transmitted to other organelles via ion flows along elements of the cytoskeleton. An evolutionary constraint to the ever-increasing acquisition of environmental information is the cost of doing so. One solution to this trade-off is the evolution of information conduits that minimize signal loss during transmission. Since the structures of these conduits are encoded in the genome, evolution of macromolecular conduits that minimize signal loss is linked to and, in fact, governed by a universal principle, termed extreme physical information (EPI). Mathematical analysis of information dynamics based on the flow of ions through membrane channels and along wire-like cytoskeleton macromolecules fulfills the EPI principle. Thus, the empirically derived model of evolution by natural selection, although uniquely applicable to living systems, is theoretically grounded in a universal principle that can also be used to derive the laws of physics. Finally, if minimization of signal loss is a mechanism to overcome energy constraints, the model predicts increasing information and associated complexity are closely linked to increased efficiency of energy production or improved substrate acquisition.

The Computational Boundary of a "Self": Developmental Bioelectricity Drives Multicellularity and Scale-Free Cognition

All epistemic agents physically consist of parts that must somehow comprise an integrated cognitive self. Biological individuals consist of subunits (organs, cells, and molecular networks) that are themselves complex and competent in their own native contexts. How do coherent biological Individuals result from the activity of smaller sub-agents? To understand the evolution and function of metazoan creatures' bodies and minds, it is essential to conceptually explore the origin of multicellularity and the scaling of the basal cognition of individual cells into a coherent larger organism. In this article, I synthesize ideas in cognitive science, evolutionary biology, and developmental physiology toward a hypothesis about the origin of Individuality: "Scale-Free Cognition." I propose a fundamental definition of an Individual based on the ability to pursue goals at an appropriate level of scale and organization and suggest a formalism for defining and comparing the cognitive capacities of highly diverse types of agents. Any Self is demarcated by a computational surface - the spatio-temporal boundary of events that it can measure, model, and try to affect. This surface sets a functional boundary - a cognitive "light cone" which defines the scale and limits of its cognition. I hypothesize that higher level goal-directed activity and agency, resulting in larger cognitive boundaries, evolve from the primal homeostatic drive of living things to reduce stress - the difference between current conditions and life-optimal conditions. The mechanisms of developmental bioelectricity - the ability of all cells to form electrical networks that process information - suggest a plausible set of gradual evolutionary steps that naturally lead from physiological homeostasis in single cells to memory, prediction, and ultimately complex cognitive agents, via scale-up of the basic drive of infotaxis. Recent data on the molecular mechanisms of pre-neural bioelectricity suggest a model of how increasingly sophisticated cognitive functions emerge smoothly from cell-cell communication used to guide embryogenesis and regeneration. This set of hypotheses provides a novel perspective on numerous phenomena, such as cancer, and makes several unique, testable predictions for interdisciplinary research that have implications not only for evolutionary developmental biology but also for biomedicine and perhaps artificial intelligence and exobiology.

Probing complexity: thermodynamics and computational mechanics approaches to origins studies

This paper proposes new avenues for origins research that apply modern concepts from stochastic thermodynamics, information thermodynamics and complexity science. Most approaches to the emergence of life prioritize certain compounds, reaction pathways, environments or phenomena. What they all have in common is the objective of reaching a state that is recognizably alive, usually positing the need for an evolutionary process. As with life itself, this correlates with a growth in the complexity of the system over time. Complexity often takes the form of an intuition or a proxy for a phenomenon that defies complete understanding. However, recent progress in several theoretical fields allows the rigorous computation of complexity. We thus propose that measurement and control of the complexity and information content of origins-relevant systems can provide novel insights that are absent in other approaches. Since we have no guarantee that the earliest forms of life (or alien life) used the same materials and processes as extant life, an appeal to complexity and information processing provides a more objective and agnostic approach to the search for life's beginnings. This paper gives an accessible overview of the three relevant branches of modern thermodynamics. These frameworks are not commonly applied in origins studies, but are ideally suited to the analysis of such non-equilibrium systems. We present proposals for the application of these concepts in both theoretical and experimental origins settings.

The plant body as a network of semi-autonomous agents: a review

Plants can solve amazingly difficult tasks while adjusting their growth and development to the environment. They can explore and exploit several resources simultaneously, even when the distributions of these vary in space and time. The systematic study of plant behaviour goes back to Darwin's book The power of movement in plants. Current research has highlighted that modularity is a key to understanding plant behaviour, as the production, functional specialization and death of modules enable the plant to adjust its movement to the environment. The adjustment is assisted by a flow of information and resources among the modules. Experiments have yielded many results about these processes in various plant species. Theoretical research, however, has lagged behind the empirical studies, possibly owing to the lack of a proper modelling framework that could encompass the high number of components and interactions. In this paper, I propose such a framework on the basis of network theory, viewing the plant as a group of connected, semi-autonomous agents. I review some characteristic plant responses to the environment through changing the states of agents and/or links. I also point out some unexplored areas, in which a dialogue between plant science and network theory could be mutually inspiring. This article is part of the theme issue 'Liquid brains, solid brains: How distributed cognitive architectures process information'.

Evolutionary aspects of reservoir computing

Reservoir computing (RC) is a powerful computational paradigm that allows high versatility with cheap learning. While other artificial intelligence approaches need exhaustive resources to specify their inner workings, RC is based on a reservoir with highly nonlinear dynamics that does not require a fine tuning of its parts. These dynamics project input signals into high-dimensional spaces, where training linear readouts to extract input features is vastly simplified. Thus, inexpensive learning provides very powerful tools for decision-making, controlling dynamical systems, classification, etc. RC also facilitates solving multiple tasks in parallel, resulting in a high throughput. Existing literature focuses on applications in artificial intelligence and neuroscience. We review this literature from an evolutionary perspective. RC's versatility makes it a great candidate to solve outstanding problems in biology, which raises relevant questions. Is RC as abundant in nature as its advantages should imply? Has it evolved? Once evolved, can it be easily sustained? Under what circumstances? (In other words, is RC an evolutionarily stable computing paradigm?) To tackle these issues, we introduce a conceptual morphospace that would map computational selective pressures that could select for or against RC and other computing paradigms. This guides a speculative discussion about the questions above and allows us to propose a solid research line that brings together computation and evolution with RC as test model of the proposed hypotheses. This article is part of the theme issue 'Liquid brains, solid brains: How distributed cognitive architectures process information'.