A taxonomy for artificial embryogeny

Stress sharing as cognitive glue for collective intelligences: A computational model of stress as a coordinator for morphogenesis

Individual cells have numerous competencies in physiological and metabolic spaces. However, multicellular collectives can reliably navigate anatomical morphospace towards much larger, reliable endpoints. Understanding the robustness and control properties of this process is critical for evolutionary developmental biology, bioengineering, and regenerative medicine. One mechanism that has been proposed for enabling individual cells to coordinate toward specific morphological outcomes is the sharing of stress (where stress is a physiological parameter that reflects the current amount of error in the context of a homeostatic loop). Here, we construct and analyze a multiscale agent-based model of morphogenesis in which we quantitatively examine the impact of stress sharing on the ability to reach target morphology. We found that stress sharing improves the morphogenetic efficiency of multicellular collectives; populations with stress sharing reached anatomical targets faster. Moreover, stress sharing influenced the future fate of distant cells in the multi-cellular collective, enhancing cells' movement and their radius of influence, consistent with the hypothesis that stress sharing works to increase cohesiveness of collectives. During development, anatomical goal states could not be inferred from observation of stress states, revealing the limitations of knowledge of goals by an extern observer outside the system itself. Taken together, our analyses support an important role for stress sharing in natural and engineered systems that seek robust large-scale behaviors to emerge from the activity of their competent components.

Evolutionary Implications of Self-Assembling Cybernetic Materials with Collective Problem-Solving Intelligence at Multiple Scales

In recent years, the scientific community has increasingly recognized the complex multi-scale competency architecture (MCA) of biology, comprising nested layers of active homeostatic agents, each forming the self-orchestrated substrate for the layer above, and, in turn, relying on the structural and functional plasticity of the layer(s) below. The question of how natural selection could give rise to this MCA has been the focus of intense research. Here, we instead investigate the effects of such decision-making competencies of MCA agential components on the process of evolution itself, using in silico neuroevolution experiments of simulated, minimal developmental biology. We specifically model the process of morphogenesis with neural cellular automata (NCAs) and utilize an evolutionary algorithm to optimize the corresponding model parameters with the objective of collectively self-assembling a two-dimensional spatial target pattern (reliable morphogenesis). Furthermore, we systematically vary the accuracy with which the uni-cellular agents of an NCA can regulate their cell states (simulating stochastic processes and noise during development). This allows us to continuously scale the agents' competency levels from a direct encoding scheme (no competency) to an MCA (with perfect reliability in cell decision executions). We demonstrate that an evolutionary process proceeds much more rapidly when evolving the functional parameters of an MCA compared to evolving the target pattern directly. Moreover, the evolved MCAs generalize well toward system parameter changes and even modified objective functions of the evolutionary process. Thus, the adaptive problem-solving competencies of the agential parts in our NCA-based in silico morphogenesis model strongly affect the evolutionary process, suggesting significant functional implications of the near-ubiquitous competency seen in living matter.

Information integration during bioelectric regulation of morphogenesis of the embryonic frog brain

Spatiotemporal patterns of cellular resting potential regulate several aspects of development. One key aspect of the bioelectric code is that transcriptional and morphogenetic states are determined not by local, single-cell, voltage levels but by specific distributions of voltage across cell sheets. We constructed and analyzed a minimal dynamical model of collective gene expression in cells based on inputs of multicellular voltage patterns. Causal integration analysis revealed a higher-order mechanism by which information about the voltage pattern was spatiotemporally integrated into gene activity, as well as a division of labor among and between the bioelectric and genetic components. We tested and confirmed predictions of this model in a system in which bioelectric control of morphogenesis regulates gene expression and organogenesis: the embryonic brain of the frog Xenopus laevis. This study demonstrates that machine learning and computational integration approaches can advance our understanding of the information-processing underlying morphogenetic decision-making, with a potential for other applications in developmental biology and regenerative medicine.

Cellular Competency during Development Alters Evolutionary Dynamics in an Artificial Embryogeny Model

Biological genotypes do not code directly for phenotypes; developmental physiology is the control layer that separates genomes from capacities ascertained by selection. A key aspect is cellular competency, since cells are not passive materials but descendants of unicellular organisms with complex context-sensitive behavioral capabilities. To probe the effects of different degrees of cellular competency on evolutionary dynamics, we used an evolutionary simulation in the context of minimal artificial embryogeny. Virtual embryos consisted of a single axis of positional information values provided by cells' 'structural genes', operated upon by an evolutionary cycle in which embryos' fitness was proportional to monotonicity of the axial gradient. Evolutionary dynamics were evaluated in two modes: hardwired development (genotype directly encodes phenotype), and a more realistic mode in which cells interact prior to evaluation by the fitness function ("regulative" development). We find that even minimal ability of cells with to improve their position in the embryo results in better performance of the evolutionary search. Crucially, we observed that increasing the behavioral competency masks the raw fitness encoded by structural genes, with selection favoring improvements to its developmental problem-solving capacities over improvements to its structural genome. This suggests the existence of a powerful ratchet mechanism: evolution progressively becomes locked in to improvements in the intelligence of its agential substrate, with reduced pressure on the structural genome. This kind of feedback loop in which evolution increasingly puts more effort into the developmental software than perfecting the hardware explains the very puzzling divergence of genome from anatomy in species like planaria. In addition, it identifies a possible driver for scaling intelligence over evolutionary time, and suggests strategies for engineering novel systems in silico and in bioengineering.

Neuro-Evolutionary Direct Policy Search for Multiobjective Optimal Control

Direct policy search (DPS) is emerging as one of the most effective and widely applied reinforcement learning (RL) methods to design optimal control policies for multiobjective Markov decision processes (MOMDPs). Traditionally, DPS defines the control policy within a preselected functional class and searches its optimal parameterization with respect to a given set of objectives. The functional class should be tailored to the problem at hand and its selection is crucial, as it determines the search space within which solutions can be found. In MOMDPs problems, a different objective tradeoff determines a different fitness landscape, requiring a tradeoff-dynamic functional class selection. Yet, in state-of-the-art applications, the policy class is generally selected a priori and kept constant across the multidimensional objective space. In this work, we present a novel policy search routine called neuro-evolutionary multiobjective DPS (NEMODPS), which extends the DPS problem formulation to conjunctively search the policy functional class and its parameterization in a hyperspace containing policy architectures and coefficients. NEMODPS begins with a population of minimally structured approximating networks and progressively builds more sophisticated architectures by topological and parametrical mutation and crossover, and selection of the fittest individuals concerning multiple objectives. We tested NEMODPS for the problem of designing the control policy of a multipurpose water system. Numerical results show that the tradeoff-dynamic structural and parametrical policy search of NEMODPS is consistent across multiple runs, and outperforms the solutions designed via traditional DPS with predefined policy topologies.

Real-World Robot Evolution: Why Would it (not) Work?

This paper takes a critical look at the concept of real-world robot evolution discussing specific challenges for making it practicable. After a brief review of the state of the art several enablers are discussed in detail. It is noted that sample efficient evolution is one of the key prerequisites and there are various promising directions towards this in different stages of maturity, including learning as part of the evolutionary system, genotype filtering, and hybridizing real-world evolution with simulations in a new way. Furthermore, it is emphasized that an evolutionary system that works in the real world needs robots that work in the real world. Obvious as it may seem, to achieve this significant complexification of the robots and their tasks is needed compared to the current practice. Finally, the importance of not only building but also understanding evolving robot systems is emphasised, stating that in order to have the technology work we also need the science behind it.

Synthetic living machines: A new window on life

Increased control of biological growth and form is an essential gateway to transformative medical advances. Repairing of birth defects, restoring lost or damaged organs, normalizing tumors, all depend on understanding how cells cooperate to make specific, functional large-scale structures. Despite advances in molecular genetics, significant gaps remain in our understanding of the meso-scale rules of morphogenesis. An engineering approach to this problem is the creation of novel synthetic living forms, greatly extending available model systems beyond evolved plant and animal lineages. Here, we review recent advances in the emerging field of synthetic morphogenesis, the bioengineering of novel multicellular living bodies. Emphasizing emergent self-organization, tissue-level guided self-assembly, and active functionality, this work is the essential next generation of synthetic biology. Aside from useful living machines for specific functions, the rational design and analysis of new, coherent anatomies will greatly increase our understanding of foundational questions in evolutionary developmental and cell biology.

A Brain-Inspired Framework for Evolutionary Artificial General Intelligence

From the medical field to agriculture, from energy to transportation, every industry is going through a revolution by embracing artificial intelligence (AI); nevertheless, AI is still in its infancy. Inspired by the evolution of the human brain, this article demonstrates a novel method and framework to synthesize an artificial brain with cognitive abilities by taking advantage of the same process responsible for the growth of the biological brain called "neuroembryogenesis." This framework shares some of the key behavioral aspects of the biological brain, such as spiking neurons, neuroplasticity, neuronal pruning, and excitatory and inhibitory interactions between neurons, together making it capable of learning and memorizing. One of the highlights of the proposed design is its potential to incrementally improve itself over generations based on system performance, using genetic algorithms. A proof of concept at the end of this article demonstrates how a simplified implementation of the human visual cortex using the proposed framework is capable of character recognition. Our framework is open source, and the code is shared with the scientific community at http://www.feagi.org.

Evolving the Behavior of Machines: From Micro to Macroevolution

Evolution gave rise to creatures that are arguably more sophisticated than the greatest human-designed systems. This feat has inspired computer scientists since the advent of computing and led to optimization tools that can evolve complex neural networks for machines-an approach known as "neuroevolution." After a few successes in designing evolvable representations for high-dimensional artifacts, the field has been recently revitalized by going beyond optimization: to many, the wonder of evolution is less in the perfect optimization of each species than in the creativity of such a simple iterative process, that is, in the diversity of species. This modern view of artificial evolution is moving the field away from microevolution, following a fitness gradient in a niche, to macroevolution, filling many niches with highly different species. It already opened promising applications, like evolving gait repertoires, video game levels for different tastes, and diverse designs for aerodynamic bikes.

New Biological Morphogenetic Methods for Evolutionary Design of Robot Bodies

We present some currently unused morphogenetic mechanisms from evolutionary biology and guidelines for transfer to evolutionary robotics. (1) DNA patterns providing mutation of mutability, lead to canalization of evolvable bauplans, via kin selection. (2) Morphogenetic mechanisms (i) Epigenetic cell lines provide functional cell types, and identification of cell descent. (ii) Local anatomical coordinates based on diffusion of morphogens, facilitate evolvable genetic parameterizations of complex phenotypes (iii) Remodeling in response to mechanical forces facilitates robust production of well-integrated phenotypes of greater complexity than the genome. An approach is proposed for the tractable application of mutation-of-mutability and morphogenetic mechanisms in evolutionary robotics. The purpose of these methods, is to facilitate production of robot mechanisms of the subtlety, efficiency, and efficacy of the musculoskeletal and dermal systems of animals.

OMNIREP: Originating Meaning by Coevolving Encodings and Representations

A major effort in the practice of evolutionary computation (EC) goes into deciding how to represent individuals in the evolving population. This task is actually composed of two subtasks: defining a data structure that is the representation and defining the encoding that enables to interpret the representation. In this paper we employ a coevolutionary algorithm-dubbed omnirep-to discover both a representation and an encoding that solve a particular problem of interest. We describe four experiments that provide a proof-of-concept of omnirep's essential merit. We think that the proposed methodology holds potential as a problem solver and also as an exploratory medium when scouting for good representations.

Why We Do Not Evolve Software? Analysis of Evolutionary Algorithms

In this article, we review the state-of-the-art results in evolutionary computation and observe that we do not evolve nontrivial software from scratch and with no human intervention. A number of possible explanations are considered, but we conclude that computational complexity of the problem prevents it from being solved as currently attempted. A detailed analysis of necessary and available computational resources is provided to support our findings.

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifelong learning. The interplay of these elements leads to the emergence of biological intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) employ simulated evolution in-silico to breed plastic neural networks with the aim to autonomously design and create learning systems. EPANN experiments evolve networks that include both innate properties and the ability to change and learn in response to experiences in different environments and problem domains. EPANNs' aims include autonomously creating learning systems, bootstrapping learning from scratch, recovering performance in unseen conditions, testing the computational advantages of particular neural components, and deriving hypotheses on the emergence of biological learning. Thus, EPANNs may include a large variety of different neuron types and dynamics, network architectures, plasticity rules, and other factors. While EPANNs have seen considerable progress over the last two decades, current scientific and technological advances in artificial neural networks are setting the conditions for radically new approaches and results. Exploiting the increased availability of computational resources and of simulation environments, the often challenging task of hand-designing learning neural networks could be replaced by more autonomous and creative processes. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and possible developments are presented.

Evolving Soft Locomotion in Aquatic and Terrestrial Environments: Effects of Material Properties and Environmental Transitions

Designing soft robots poses considerable challenges; automated design approaches may be particularly appealing in this field, as they promise to optimize complex multimaterial machines with very little or no human intervention. Evolutionary soft robotics is concerned with the application of optimization algorithms inspired by natural evolution to let soft robots (both their morphologies and controllers) spontaneously evolve within physically realistic simulated environments, figuring out how to satisfy a set of objectives defined by human designers. In this article, a powerful evolutionary system is put in place to perform a broad investigation on the free-form evolution of simulated walking and swimming soft robots in different environments. Three sets of experiments are reported, tackling different aspects of the evolution of soft locomotion. The first two explore the effects of different material properties on the evolution of terrestrial and aquatic soft locomotion: particularly, we show how different materials lead to the evolution of different morphologies, behaviors, and energy-performance trade-offs. It is found that within our simplified physics world, stiffer robots evolve more sophisticated and effective gaits and morphologies on land, while softer ones tend to perform better in water. The third set of experiments starts investigating the effect and potential benefits of major environmental transitions (land↔water) during evolution. Results provide interesting morphological exaptation phenomena and point out a potential asymmetry between land→water and water→land transitions: while the first type of transition appears to be detrimental, the second one seems to have some beneficial effects.

The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System

Many believe that an essential component for the discovery of the tremendous diversity in natural organisms was the evolution of evolvability, whereby evolution speeds up its ability to innovate by generating a more adaptive pool of offspring. One hypothesized mechanism for evolvability is developmental canalization, wherein certain dimensions of variation become more likely to be traversed and others are prevented from being explored (e.g., offspring tend to have similar-size legs, and mutations affect the length of both legs, not each leg individually). While ubiquitous in nature, canalization is rarely reported in computational simulations of evolution, which deprives us of in silico examples of canalization to study and raises the question of which conditions give rise to this form of evolvability. Answering this question would shed light on why such evolvability emerged naturally, and it could accelerate engineering efforts to harness evolution to solve important engineering challenges. In this article, we reveal a unique system in which canalization did emerge in computational evolution. We document that genomes entrench certain dimensions of variation that were frequently explored during their evolutionary history. The genetic representation of these organisms also evolved to be more modular and hierarchical than expected by chance, and we show that these organizational properties correlate with increased fitness. Interestingly, the type of computational evolutionary experiment that produced this evolvability was very different from traditional digital evolution in that there was no objective, suggesting that open-ended, divergent evolutionary processes may be necessary for the evolution of evolvability.

Improving HybrID: How to best combine indirect and direct encoding in evolutionary algorithms

Many challenging engineering problems are regular, meaning solutions to one part of a problem can be reused to solve other parts. Evolutionary algorithms with indirect encoding perform better on regular problems because they reuse genomic information to create regular phenotypes. However, on problems that are mostly regular, but contain some irregularities, which describes most real-world problems, indirect encodings struggle to handle the irregularities, hurting performance. Direct encodings are better at producing irregular phenotypes, but cannot exploit regularity. An algorithm called HybrID combines the best of both: it first evolves with indirect encoding to exploit problem regularity, then switches to direct encoding to handle problem irregularity. While HybrID has been shown to outperform both indirect and direct encoding, its initial implementation required the manual specification of when to switch from indirect to direct encoding. In this paper, we test two new methods to improve HybrID by eliminating the need to manually specify this parameter. Auto-Switch-HybrID automatically switches from indirect to direct encoding when fitness stagnates. Offset-HybrID simultaneously evolves an indirect encoding with directly encoded offsets, eliminating the need to switch. We compare the original HybrID to these alternatives on three different problems with adjustable regularity. The results show that both Auto-Switch-HybrID and Offset-HybrID outperform the original HybrID on different types of problems, and thus offer more tools for researchers to solve challenging problems. The Offset-HybrID algorithm is particularly interesting because it suggests a path forward for automatically and simultaneously combining the best traits of indirect and direct encoding.

Developmental Design of Synthetic Bacterial Architectures by Morphogenetic Engineering

Synthetic biology is an emerging scientific field that promotes the standardized manufacturing of biological components without natural equivalents. Its goal is to create artificial living systems that can meet various needs in health care or energy domains. While most works are focused on the individual bacterium as a chemical reactor, our project, SynBioTIC, addresses a novel and more complex challenge: shape engineering; that is, the redesign of natural morphogenesis toward a new kind of developmental 3D printing. Potential applications include organ growth, natural computing in biocircuits, or future vegetal houses. To create in silico multicellular organisms that exhibit specific shapes, we construe their development as an iterative process combining fundamental collective phenomena such as homeostasis, patterning, segmentation, and limb growth. Our numerical experiments rely on the existing Escherichia coli simulator Gro, a physicochemical computation platform offering reaction-diffusion and collision dynamics solvers. The synthetic bioware of our model executes a set of rules, or genome, in each cell. Cells can differentiate into several predefined types associated with specific actions (divide, emit signal, detect signal, die). Transitions between types are triggered by conditions involving internal and external sensors that detect various protein levels inside and around the cell. Indirect communication between bacteria is relayed by morphogen diffusion and the mechanical constraints of 2D packing. Starting from a single bacterium, the overall architecture emerges in a purely endogenous fashion through a series of developmental stages, inlcuding proliferation, differentiation, morphogen diffusion, and synchronization. The genome can be parametrized to control the growth and features of appendages individually. As exemplified by the L and T shapes that we obtain, certain precursor cells can be inhibited while others can create limbs of varying size (divergence of the homology). Such morphogenetic phenotypes open the way to more complex shapes made of a recursive array of core bodies and limbs and, most importantly, to an evolutionary developmental exploration of unplanned functional forms.

The Evolutionary Origins of Hierarchy

Hierarchical organization-the recursive composition of sub-modules-is ubiquitous in biological networks, including neural, metabolic, ecological, and genetic regulatory networks, and in human-made systems, such as large organizations and the Internet. To date, most research on hierarchy in networks has been limited to quantifying this property. However, an open, important question in evolutionary biology is why hierarchical organization evolves in the first place. It has recently been shown that modularity evolves because of the presence of a cost for network connections. Here we investigate whether such connection costs also tend to cause a hierarchical organization of such modules. In computational simulations, we find that networks without a connection cost do not evolve to be hierarchical, even when the task has a hierarchical structure. However, with a connection cost, networks evolve to be both modular and hierarchical, and these networks exhibit higher overall performance and evolvability (i.e. faster adaptation to new environments). Additional analyses confirm that hierarchy independently improves adaptability after controlling for modularity. Overall, our results suggest that the same force-the cost of connections-promotes the evolution of both hierarchy and modularity, and that these properties are important drivers of network performance and adaptability. In addition to shedding light on the emergence of hierarchy across the many domains in which it appears, these findings will also accelerate future research into evolving more complex, intelligent computational brains in the fields of artificial intelligence and robotics.

Evolutionary Growth of Genome Representations on Artificial Cellular Organisms with Indirect Encodings

Evolutionary design targets systems of continuously increasing complexity. Thus, indirect developmental mappings are often a necessity. Varying the amount of genotype information changes the cardinality of the mapping, which in turn affects the developmental process. An open question is how to find the genotype size and representation in which a developmental solution would fit. A restricted pool of genes may not be large enough to encode a solution or may need complex heuristics to find a realistic size. On the other hand, using the whole set of possible regulatory combinations may be intractable. In nature, the genomes of biological organisms are not fixed in size; they slowly evolve and acquire new genes by random gene duplications. Such incremental growth of genome information can be beneficial also in the artificial domain. For an evolutionary and developmental (evo-devo) system based on cellular automata, we investigate an incremental evolutionary growth of genomes without any a priori knowledge on the necessary genotype size. Evolution starts with simple solutions in a low-dimensional space and incrementally increases the genotype complexity by means of gene duplication, allowing the evolution of scalable genomes that are able to adapt genetic information content while compactness and efficiency are retained. The results are consistent when the target phenotypic complexity, the geometry size, and the number of cell states are scaled up.

Open Issues in Evolutionary Robotics

One of the long-term goals in evolutionary robotics is to be able to automatically synthesize controllers for real autonomous robots based only on a task specification. While a number of studies have shown the applicability of evolutionary robotics techniques for the synthesis of behavioral control, researchers have consistently been faced with a number of issues preventing the widespread adoption of evolutionary robotics for engineering purposes. In this article, we review and discuss the open issues in evolutionary robotics. First, we analyze the benefits and challenges of simulation-based evolution and subsequent deployment of controllers versus evolution on real robotic hardware. Second, we discuss specific evolutionary computation issues that have plagued evolutionary robotics: (1) the bootstrap problem, (2) deception, and (3) the role of genomic encoding and genotype-phenotype mapping in the evolution of controllers for complex tasks. Finally, we address the absence of standard research practices in the field. We also discuss promising avenues of research. Our underlying motivation is the reduction of the current gap between evolutionary robotics and mainstream robotics, and the establishment of evolutionary robotics as a canonical approach for the engineering of autonomous robots.

Neural modularity helps organisms evolve to learn new skills without forgetting old skills

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting.

A long-standing goal in artificial intelligence (AI) is creating computational brain models (neural networks) that learn what to do in new situations. An obstacle is that agents typically learn new skills only by losing previously acquired skills. Here we test whether such forgetting is reduced by evolving modular neural networks, meaning networks with many distinct subgroups of neurons. Modularity intuitively should help because learning can be selectively turned on only in the module learning the new task. We confirm this hypothesis: modular networks have higher overall performance because they learn new skills faster while retaining old skills more. Our results suggest that one benefit of modularity in natural animal brains may be allowing learning without forgetting.

Investigating Biological Assumptions through Radical Reimplementation

An important goal in both artificial life and biology is uncovering the most general principles underlying life, which might catalyze both our understanding of life and engineering lifelike machines. While many such general principles have been hypothesized, conclusively testing them is difficult because life on Earth provides only a singular example from which to infer. To circumvent this limitation, this article formalizes an approach called radical reimplementation. The idea is to investigate an abstract biological hypothesis by intentionally reimplementing its main principles to diverge maximally from existing natural examples. If the reimplementation successfully exhibits properties resembling biology, it may support the underlying hypothesis better than an alternative example inspired more directly by nature. The approach thereby provides a principled alternative to a common tradition of defending and minimizing deviations from nature in artificial life. This work reviews examples that can be interpreted through the lens of radical reimplementation to yield potential insights into biology despite having purposely unnatural experimental setups. In this way, radical reimplementation can help renew the relevance of computational systems for investigating biological theory and can act as a practical philosophical tool to help separate the fundamental features of terrestrial biology from the epiphenomenal.

Unshackling evolution

In 1994, Karl Sims' evolved virtual creatures showed the potential of evolutionary algorithms to produce natural, complex morphologies and behaviors [30]. One might assume that nearly 20 years of improvements in computational speed and evolutionary algorithms would produce far more impressive organisms, yet the creatures evolved in the field of artificial life today are not obviously more complex, natural, or intelligent. Fig. 2 demonstrates an example of similar complexity in robots evolved 17 years apart.

A proto-architecture for innate directionally selective visual maps

Self-organizing artificial neural networks are a popular tool for studying visual system development, in particular the cortical feature maps present in real systems that represent properties such as ocular dominance (OD), orientation-selectivity (OR) and direction selectivity (DS). They are also potentially useful in artificial systems, for example robotics, where the ability to extract and learn features from the environment in an unsupervised way is important. In this computational study we explore a DS map that is already latent in a simple artificial network. This latent selectivity arises purely from the cortical architecture without any explicit coding for DS and prior to any self-organising process facilitated by spontaneous activity or training. We find DS maps with local patchy regions that exhibit features similar to maps derived experimentally and from previous modeling studies. We explore the consequences of changes to the afferent and lateral connectivity to establish the key features of this proto-architecture that support DS.

Improving the adaptability of simulated evolutionary swarm robots in dynamically changing environments

One of the important challenges in the field of evolutionary robotics is the development of systems that can adapt to a changing environment. However, the ability to adapt to unknown and fluctuating environments is not straightforward. Here, we explore the adaptive potential of simulated swarm robots that contain a genomic encoding of a bio-inspired gene regulatory network (GRN). An artificial genome is combined with a flexible agent-based system, representing the activated part of the regulatory network that transduces environmental cues into phenotypic behaviour. Using an artificial life simulation framework that mimics a dynamically changing environment, we show that separating the static from the conditionally active part of the network contributes to a better adaptive behaviour. Furthermore, in contrast with most hitherto developed ANN-based systems that need to re-optimize their complete controller network from scratch each time they are subjected to novel conditions, our system uses its genome to store GRNs whose performance was optimized under a particular environmental condition for a sufficiently long time. When subjected to a new environment, the previous condition-specific GRN might become inactivated, but remains present. This ability to store 'good behaviour' and to disconnect it from the novel rewiring that is essential under a new condition allows faster re-adaptation if any of the previously observed environmental conditions is reencountered. As we show here, applying these evolutionary-based principles leads to accelerated and improved adaptive evolution in a non-stable environment.

An efficient automated parameter tuning framework for spiking neural networks

As the desire for biologically realistic spiking neural networks (SNNs) increases, tuning the enormous number of open parameters in these models becomes a difficult challenge. SNNs have been used to successfully model complex neural circuits that explore various neural phenomena such as neural plasticity, vision systems, auditory systems, neural oscillations, and many other important topics of neural function. Additionally, SNNs are particularly well-adapted to run on neuromorphic hardware that will support biological brain-scale architectures. Although the inclusion of realistic plasticity equations, neural dynamics, and recurrent topologies has increased the descriptive power of SNNs, it has also made the task of tuning these biologically realistic SNNs difficult. To meet this challenge, we present an automated parameter tuning framework capable of tuning SNNs quickly and efficiently using evolutionary algorithms (EA) and inexpensive, readily accessible graphics processing units (GPUs). A sample SNN with 4104 neurons was tuned to give V1 simple cell-like tuning curve responses and produce self-organizing receptive fields (SORFs) when presented with a random sequence of counterphase sinusoidal grating stimuli. A performance analysis comparing the GPU-accelerated implementation to a single-threaded central processing unit (CPU) implementation was carried out and showed a speedup of 65× of the GPU implementation over the CPU implementation, or 0.35 h per generation for GPU vs. 23.5 h per generation for CPU. Additionally, the parameter value solutions found in the tuned SNN were studied and found to be stable and repeatable. The automated parameter tuning framework presented here will be of use to both the computational neuroscience and neuromorphic engineering communities, making the process of constructing and tuning large-scale SNNs much quicker and easier.

Environmental influence on the evolution of morphological complexity in machines

Whether, when, how, and why increased complexity evolves in biological populations is a longstanding open question. In this work we combine a recently developed method for evolving virtual organisms with an information-theoretic metric of morphological complexity in order to investigate how the complexity of morphologies, which are evolved for locomotion, varies across different environments. We first demonstrate that selection for locomotion results in the evolution of organisms with morphologies that increase in complexity over evolutionary time beyond what would be expected due to random chance. This provides evidence that the increase in complexity observed is a result of a driven rather than a passive trend. In subsequent experiments we demonstrate that morphologies having greater complexity evolve in complex environments, when compared to a simple environment when a cost of complexity is imposed. This suggests that in some niches, evolution may act to complexify the body plans of organisms while in other niches selection favors simpler body plans.

The evolution of complexity, a central issue of evolutionary theory since Darwin's time, remains a controversial topic. One particular question of interest is how the complexity of an organism's body plan (morphology) is influenced by the complexity of the environment in which it evolved. Ideally, it would be desirable to perform investigations on living organisms in which environmental complexity is under experimental control, but our ability to do so in a limited timespan and in a controlled manner is severely constrained. In lieu of such studies, here we employ computer simulations capable of evolving the body plans of virtual organisms to investigate this question in silico. By evolving virtual organisms for locomotion in a variety of environments, we are able to demonstrate that selecting for locomotion causes more complex morphologies to evolve than would be expected solely due to random chance. Moreover, if increased complexity incurs a cost (as it is thought to do in biology), then more complex environments tend to lead to the evolution of more complex body plans than those that evolve in a simpler environment. This result supports the idea that the morphological complexity of organisms is influenced by the complexity of the environments in which they evolve.

On the relationships between generative encodings, regularity, and learning abilities when evolving plastic artificial neural networks

A major goal of bio-inspired artificial intelligence is to design artificial neural networks with abilities that resemble those of animal nervous systems. It is commonly believed that two keys for evolving nature-like artificial neural networks are (1) the developmental process that links genes to nervous systems, which enables the evolution of large, regular neural networks, and (2) synaptic plasticity, which allows neural networks to change during their lifetime. So far, these two topics have been mainly studied separately. The present paper shows that they are actually deeply connected. Using a simple operant conditioning task and a classic evolutionary algorithm, we compare three ways to encode plastic neural networks: a direct encoding, a developmental encoding inspired by computational neuroscience models, and a developmental encoding inspired by morphogen gradients (similar to HyperNEAT). Our results suggest that using a developmental encoding could improve the learning abilities of evolved, plastic neural networks. Complementary experiments reveal that this result is likely the consequence of the bias of developmental encodings towards regular structures: (1) in our experimental setup, encodings that tend to produce more regular networks yield networks with better general learning abilities; (2) whatever the encoding is, networks that are the more regular are statistically those that have the best learning abilities.

Growing and evolving soft robots

Completely soft and flexible robots offer to revolutionize fields ranging from search and rescue to endoscopic surgery. One of the outstanding challenges in this burgeoning field is the chicken-and-egg problem of body-brain design: Development of locomotion requires the preexistence of a locomotion-capable body, and development of a location-capable body requires the preexistence of a locomotive gait. This problem is compounded by the high degree of coupling between the material properties of a soft body (such as stiffness or damping coefficients) and the effectiveness of a gait. This article synthesizes four years of research into soft robotics, in particular describing three approaches to the co-discovery of soft robot morphology and control. In the first, muscle placement and firing patterns are coevolved for a fixed body shape with fixed material properties. In the second, the material properties of a simulated soft body coevolve alongside locomotive gaits, with body shape and muscle placement fixed. In the third, a developmental encoding is used to scalably grow elaborate soft body shapes from a small seed structure. Considerations of the simulation time and the challenges of physically implementing soft robots in the real world are discussed.

An enhanced hypercube-based encoding for evolving the placement, density, and connectivity of neurons

Intelligence in nature is the product of living brains, which are themselves the product of natural evolution. Although researchers in the field of neuroevolution (NE) attempt to recapitulate this process, artificial neural networks (ANNs) so far evolved through NE algorithms do not match the distinctive capabilities of biological brains. The recently introduced hypercube-based neuroevolution of augmenting topologies (HyperNEAT) approach narrowed this gap by demonstrating that the pattern of weights across the connectivity of an ANN can be generated as a function of its geometry, thereby allowing large ANNs to be evolved for high-dimensional problems. Yet the positions and number of the neurons connected through this approach must be decided a priori by the user and, unlike in living brains, cannot change during evolution. Evolvable-substrate HyperNEAT (ES-HyperNEAT), introduced in this article, addresses this limitation by automatically deducing the node geometry from implicit information in the pattern of weights encoded by HyperNEAT, thereby avoiding the need to evolve explicit placement. This approach not only can evolve the location of every neuron in the network, but also can represent regions of varying density, which means resolution can increase holistically over evolution. ES-HyperNEAT is demonstrated through multi-task, maze navigation, and modular retina domains, revealing that the ANNs generated by this new approach assume natural properties such as neural topography and geometric regularity. Also importantly, ES-HyperNEAT's compact indirect encoding can be seeded to begin with a bias toward a desired class of ANN topographies, which facilitates the evolutionary search. The main conclusion is that ES-HyperNEAT significantly expands the scope of neural structures that evolution can discover.

Evolution and analysis of genetic networks for stable cellular growth and regeneration

A computational model is presented that simulates stable growth of cellular structures that are in some cases capable of regeneration. In the model, cellular growth is governed by a gene regulatory network. By evolving the parameters and structure of the genetic network using a modified evolution strategy, a dynamically stable state can be achieved in the developmental process, where cell proliferation and cell apoptosis reach an equilibrium. The results of evolution with different setups in fitness evaluation during the development are compared with respect to their regeneration capability as well as their gene regulatory network structure. Network motifs responsible for stable growth and regeneration that emerged from the evolution are also analyzed. We expect that our findings can help to gain a better understanding of the process of growth and regeneration inspired by biological systems, in order to solve complex engineering problems, such as the design of self-healing materials.

Tackling learning intractability through topological organization and regulation of cortical networks

A key challenge in evolving control systems for robots using neural networks is training tractability. Evolving monolithic fixed topology neural networks is shown to be intractable with limited supervision in high dimensional search spaces. Common strategies to overcome this limitation are to provide more supervision by encouraging particular solution strategies, manually decomposing the task and segmenting the search space and network. These strategies require a supervisor with domain knowledge and may not be feasible for difficult tasks where novel concepts are required. The alternate strategy is to use self-organized task decomposition to solve difficult tasks with limited supervision. The artificial neural tissue (ANT) approach presented here uses self-organized task decomposition to solve tasks. ANT inspired by neurobiology combines standard neural networks with a novel wireless signaling scheme modeling chemical diffusion of neurotransmitters. These chemicals are used to dynamically activate and inhibit wired network of neurons using a coarse-coding framework. Using only a global fitness function that does not encourage a predefined solution, modular networks of neurons are shown to self-organize and perform task decomposition. This approach solves the sign-following task found to be intractable with conventional fixed and variable topology networks. In this paper, key attributes of the ANT architecture that perform self-organized task decomposition are shown. The architecture is robust and scalable to number of neurons, synaptic connections, and initialization parameters.

Evolving neural fields for problems with large input and output spaces

We have developed an extension of the NEAT neuroevolution method, called NEATfields, to solve problems with large input and output spaces. The NEATfields method is a multilevel neuroevolution method using externally specified design patterns. Its networks have three levels of architecture. The highest level is a NEAT-like network of neural fields. The intermediate level is a field of identical subnetworks, called field elements, with a two-dimensional topology. The lowest level is a NEAT-like subnetwork of neurons. The topology and connection weights of these networks are evolved with methods derived from the NEAT method. Evolution is provided with further design patterns to enable information flow between field elements, to dehomogenize neural fields, and to enable detection of local features. We show that the NEATfields method can solve a number of high dimensional pattern recognition and control problems, provide conceptual and empirical comparison with the state of the art HyperNEAT method, and evaluate the benefits of different design patterns.

Emergent diversity in an open-ended evolving virtual community

Understanding the dynamics of biodiversity has become an important line of research in theoretical ecology and, in particular, conservation biology. However, studying the evolution of ecological communities under traditional modeling approaches based on differential calculus requires species' characteristics to be predefined, which limits the generality of the results. An alternative but less standardized methodology relies on intensive computer simulation of evolving communities made of simple, explicitly described individuals. We study here the formation, evolution, and diversity dynamics of a community of virtual plants with a novel individual-centered model involving three different scales: the genetic, the developmental, and the physiological scales. It constitutes an original attempt at combining development, evolution, and population dynamics (based on multi-agent interactions) into one comprehensive, yet simple model. In this world, we observe that our simulated plants evolve increasingly elaborate canopies, which are capable of intercepting ever greater amounts of light. Generated morphologies vary from the simplest one-branch structure of promoter plants to a complex arborization of several hundred thousand branches in highly evolved variants. On the population scale, the heterogeneous spatial structuration of the plant community at each generation depends solely on the evolution of its component plants. Using this virtual data, the morphologies and the dynamics of diversity production were analyzed by various statistical methods, based on genotypic and phenotypic distance metrics. The results demonstrate that diversity can spontaneously emerge in a community of mutually interacting individuals under the influence of specific environmental conditions.