The influence of learning on evolution: a mathematical framework

Evolutionary Implications of Self-Assembling Cybernetic Materials with Collective Problem-Solving Intelligence at Multiple Scales

In recent years, the scientific community has increasingly recognized the complex multi-scale competency architecture (MCA) of biology, comprising nested layers of active homeostatic agents, each forming the self-orchestrated substrate for the layer above, and, in turn, relying on the structural and functional plasticity of the layer(s) below. The question of how natural selection could give rise to this MCA has been the focus of intense research. Here, we instead investigate the effects of such decision-making competencies of MCA agential components on the process of evolution itself, using in silico neuroevolution experiments of simulated, minimal developmental biology. We specifically model the process of morphogenesis with neural cellular automata (NCAs) and utilize an evolutionary algorithm to optimize the corresponding model parameters with the objective of collectively self-assembling a two-dimensional spatial target pattern (reliable morphogenesis). Furthermore, we systematically vary the accuracy with which the uni-cellular agents of an NCA can regulate their cell states (simulating stochastic processes and noise during development). This allows us to continuously scale the agents' competency levels from a direct encoding scheme (no competency) to an MCA (with perfect reliability in cell decision executions). We demonstrate that an evolutionary process proceeds much more rapidly when evolving the functional parameters of an MCA compared to evolving the target pattern directly. Moreover, the evolved MCAs generalize well toward system parameter changes and even modified objective functions of the evolutionary process. Thus, the adaptive problem-solving competencies of the agential parts in our NCA-based in silico morphogenesis model strongly affect the evolutionary process, suggesting significant functional implications of the near-ubiquitous competency seen in living matter.

Cellular Competency during Development Alters Evolutionary Dynamics in an Artificial Embryogeny Model

Biological genotypes do not code directly for phenotypes; developmental physiology is the control layer that separates genomes from capacities ascertained by selection. A key aspect is cellular competency, since cells are not passive materials but descendants of unicellular organisms with complex context-sensitive behavioral capabilities. To probe the effects of different degrees of cellular competency on evolutionary dynamics, we used an evolutionary simulation in the context of minimal artificial embryogeny. Virtual embryos consisted of a single axis of positional information values provided by cells' 'structural genes', operated upon by an evolutionary cycle in which embryos' fitness was proportional to monotonicity of the axial gradient. Evolutionary dynamics were evaluated in two modes: hardwired development (genotype directly encodes phenotype), and a more realistic mode in which cells interact prior to evaluation by the fitness function ("regulative" development). We find that even minimal ability of cells with to improve their position in the embryo results in better performance of the evolutionary search. Crucially, we observed that increasing the behavioral competency masks the raw fitness encoded by structural genes, with selection favoring improvements to its developmental problem-solving capacities over improvements to its structural genome. This suggests the existence of a powerful ratchet mechanism: evolution progressively becomes locked in to improvements in the intelligence of its agential substrate, with reduced pressure on the structural genome. This kind of feedback loop in which evolution increasingly puts more effort into the developmental software than perfecting the hardware explains the very puzzling divergence of genome from anatomy in species like planaria. In addition, it identifies a possible driver for scaling intelligence over evolutionary time, and suggests strategies for engineering novel systems in silico and in bioengineering.

Adaptability and evolution

The capacity of organisms to respond in their own lifetimes to new challenges in their environments probably appeared early in biological evolution. At present few studies have shown how such adaptability could influence the inherited characteristics of an organism's descendants. In part, this has been because organisms have been treated as passive in evolution. Nevertheless, their effects on biological evolution are likely to have been important and, when they occurred, accelerated the pace of evolution. Ways in which this might have happened have been suggested many times since the 1870s. I review these proposals and discuss their relevance to modern thought.

A simple computational model of the evolution of a communicative trait and its phenotypic plasticity

We consider a simple computational model of the evolution of a quantitative trait and its phenotypic plasticity based on directional and positive frequency-dependent selection in order to explore whether and how leaning might facilitate evolution under the dynamics that arise from communicative interactions among individuals. In the model, each individual expresses, at many different times in its lifetime, its real-valued trait depending on the probability distribution determined by its own genotypes. In communicative interactions between two individuals, the contribution of an interaction to the fitness is high when their trait values are close to each other as well as large, which represents the positive frequency-dependent and directional components of selection, respectively. The iterative interactions allow individuals to acquire a more adaptive trait pair through trial and error. Under the stochastic evolution process with the limited number of individuals, we show that learning allows the population to avoid getting stuck in the global but low optimum of the innate and individual-level fitness landscape via both aspects of the components of selection, and brings about the successful evolution by increasing the genetic variation of the population. We also analyze how such an effect of learning can be realized by measuring the degree of the two different contributions for increasing the adaptivity and similarity of communicative traits, respectively. We show that this effect of learning arises from these different types of contributions depending on the biological and environmental conditions such as the mutation rate and the duration of communicative interactions. We further show the condition for the complete genetic assimilation to occur.

A Study on Computational Efficiency and Plasticity in Baldwinian Learning

Baldwinian evolution is a hybridization of populationbased search and local refinements. Unlike in Lamarckian evolution, selection is made based on fitness improved in local refining, but refined traits are not known to offspring. For potential use in computational applications, we must investigate the Baldwinian evolutionmechanism, in term of computational cost and fitness improvement. In this paper, a set of experiments is presented, to find what is produced in Baldwinian learning. We found that, on the static landscapes involved, learning cost is paid to maintain a certain level of potential to reach good solutions, rather than to further explore on the landscape. Plasticity codes in genotypes can help in selecting appropriate parts to refine and improve search performance. However, this improvement remains limited because no learned traits are passed on, and does not enable exploration far beyond parents.

Evolution of cooperation facilitated by reinforcement learning with adaptive aspiration levels

Repeated interaction between individuals is the main mechanism for maintaining cooperation in social dilemma situations. Variants of tit-for-tat (repeating the previous action of the opponent) and the win-stay lose-shift strategy are known as strong competitors in iterated social dilemma games. On the other hand, real repeated interaction generally allows plasticity (i.e., learning) of individuals based on the experience of the past. Although plasticity is relevant to various biological phenomena, its role in repeated social dilemma games is relatively unexplored. In particular, if experience-based learning plays a key role in promotion and maintenance of cooperation, learners should evolve in the contest with nonlearners under selection pressure. By modeling players using a simple reinforcement learning model, we numerically show that learning enables the evolution of cooperation. We also show that numerically estimated adaptive dynamics appositely predict the outcome of evolutionary simulations. The analysis of the adaptive dynamics enables us to capture the obtained results as an affirmative example of the Baldwin effect, where learning accelerates the evolution to optimality.