Stochastic evolutionary dynamics on two levels

Evolutionary dynamics within and among competing groups

Biological and social systems are structured at multiple scales, and the incentives of individuals who interact in a group may diverge from the collective incentive of the group as a whole. Mechanisms to resolve this tension are responsible for profound transitions in evolutionary history, including the origin of cellular life, multicellular life, and even societies. Here, we synthesize a growing literature that extends evolutionary game theory to describe multilevel evolutionary dynamics, using nested birth-death processes and partial differential equations to model natural selection acting on competition within and among groups of individuals. We analyze how mechanisms known to promote cooperation within a single group-including assortment, reciprocity, and population structure-alter evolutionary outcomes in the presence of competition among groups. We find that population structures most conducive to cooperation in multiscale systems can differ from those most conducive within a single group. Likewise, for competitive interactions with a continuous range of strategies we find that among-group selection may fail to produce socially optimal outcomes, but it can nonetheless produce second-best solutions that balance individual incentives to defect with the collective incentives for cooperation. We conclude by describing the broad applicability of multiscale evolutionary models to problems ranging from the production of diffusible metabolites in microbes to the management of common-pool resources in human societies.

Assortment and Reciprocity Mechanisms for Promotion of Cooperation in a Model of Multilevel Selection

In the study of the evolution of cooperation, many mechanisms have been proposed to help overcome the self-interested cheating that is individually optimal in the Prisoners' Dilemma game. These mechanisms include assortative or networked social interactions, other-regarding preferences considering the payoffs of others, reciprocity rules to establish cooperation as a social norm, and multilevel selection involving simultaneous competition between individuals favoring cheaters and competition between groups favoring cooperators. In this paper, we build on recent work studying PDE replicator equations for multilevel selection to understand how within-group mechanisms of assortment, other-regarding preferences, and both direct and indirect reciprocity can help to facilitate cooperation in concert with evolutionary competition between groups. We consider a group-structured population in which interactions between individuals consist of Prisoners' Dilemma games and study the dynamics of multilevel competition determined by the payoffs individuals receive when interacting according to these within-group mechanisms. We find that the presence of each of these mechanisms acts synergistically with multilevel selection for the promotion of cooperation, decreasing the strength of between-group competition required to sustain long-time cooperation and increasing the collective payoff achieved by the population. However, we find that only other-regarding preferences allow for the achievement of socially optimal collective payoffs for Prisoners' Dilemma games in which average payoff is maximized by an intermediate mix of cooperators and defectors. For the other three mechanisms, the multilevel dynamics remain susceptible to a shadow of lower-level selection, as the collective outcome fails to exceed the payoff of the all-cooperator group.

A PDE Model for Protocell Evolution and the Origin of Chromosomes via Multilevel Selection

The evolution of complex cellular life involved two major transitions: the encapsulation of self-replicating genetic entities into cellular units and the aggregation of individual genes into a collectively replicating genome. In this paper, we formulate a minimal model of the evolution of proto-chromosomes within protocells. We model a simple protocell composed of two types of genes: a "fast gene" with an advantage for gene-level self-replication and a "slow gene" that replicates more slowly at the gene level, but which confers an advantage for protocell-level reproduction. Protocell-level replication capacity depends on cellular composition of fast and slow genes. We use a partial differential equation to describe how the composition of genes within protocells evolves over time under within-cell and between-cell competition, considering an infinite population of protocells that each contain infinitely many genes. We find that the gene-level advantage of fast replicators casts a long shadow on the multilevel dynamics of protocell evolution: no level of between-protocell competition can produce coexistence of the fast and slow replicators when the two genes are equally needed for protocell-level reproduction. By introducing a "dimer replicator" consisting of a linked pair of the slow and fast genes, we show analytically that coexistence between the two genes can be promoted in pairwise multilevel competition between fast and dimer replicators, and provide numerical evidence for coexistence in trimorphic competition between fast, slow, and dimer replicators. Our results suggest that dimerization, or the formation of a simple chromosome-like dimer replicator, can help to overcome the shadow of lower-level selection and work in concert with deterministic multilevel selection in protocells featuring high gene copy number to allow for the coexistence of two genes that are complementary at the protocell level but compete at the level of individual gene-level replication. These results for the PDE model complement existing results on the benefits of dimerization in the case of low genetic copy number, for which it has been shown that genetic linkage can help to overcome the stochastic loss of necessary genetic templates.

Long-time behavior of a PDE replicator equation for multilevel selection in group-structured populations

In many biological systems, natural selection acts simultaneously on multiple levels of organization. This scenario typically presents an evolutionary conflict between the incentive of individuals to cheat and the collective incentive to establish cooperation within a group. Generalizing previous work on multilevel selection in evolutionary game theory, we consider a hyperbolic PDE model of a group-structured population, in which members within a single group compete with each other for individual-level replication; while the group also competes against other groups for group-level replication. We derive a threshold level of the relative strength of between-group competition such that defectors take over the population below the threshold while cooperation persists in the long-time population above the threshold. Under stronger assumptions on the initial distribution of group compositions, we further prove that the population converges to a steady state density supporting cooperation for between-group selection strength above the threshold. We further establish long-time bounds on the time-average of the collective payoff of the population, showing that the long-run population cannot outperform the payoff of a full-cooperator group even in the limit of infinitely-strong between-group competition. When the group replication rate is maximized by an intermediate level of within-group cooperation, individual-level selection casts a long shadow on the dynamics of multilevel selection: no level of between-group competition can erase the effects of the individual incentive to defect. We further extend our model to study the case of multiple types of groups, showing how the games that groups play can coevolve with the level of cooperation.

Toward a Universal Model for Spatially Structured Populations

A key question in evolution is how likely a mutant is to take over. This depends on natural selection and on stochastic fluctuations. Population spatial structure can impact mutant fixation probabilities. We introduce a model for structured populations on graphs that generalizes previous ones by making migrations independent of birth and death. We demonstrate that by tuning migration asymmetry, the star graph transitions from amplifying to suppressing natural selection. The results from our model are universal in the sense that they do not hinge on a modeling choice of microscopic dynamics or update rules. Instead, they depend on migration asymmetry, which can be experimentally tuned and measured.

Analysis of Multilevel Replicator Dynamics for General Two-Strategy Social Dilemma

Here, we consider a game-theoretic model of multilevel selection in which individuals compete based on their payoff and groups also compete based on the average payoff of group members. Our focus is on multilevel social dilemmas: games in which individuals are best off cheating, while groups of individuals do best when composed of many cooperators. We analyze the dynamics of the two-level replicator dynamics, a nonlocal hyperbolic PDE describing deterministic birth-death dynamics for both individuals and groups. While past work on such multilevel dynamics has restricted attention to scenarios with exactly solvable within-group dynamics, we use comparison principles and an invariant property of the tail of the population distribution to extend our analysis to all possible two-player, two-strategy social dilemmas. In the Stag-Hunt and similar games with coordination thresholds, we show that any amount of between-group competition allows for fixation of cooperation in the population. For the prisoners' dilemma and Hawk-Dove game, we characterize the threshold level of between-group selection dividing a regime in which the population converges to a delta function at the equilibrium of the within-group dynamics from a regime in which between-group competition facilitates the existence of steady-state densities supporting greater levels of cooperation. In particular, we see that the threshold selection strength and average payoff at steady state depend on a tug-of-war between the individual-level incentive to be a defector in a many-cooperator group and the group-level incentive to have many cooperators over many defectors. We also find that lower-level selection casts a long shadow: If groups are best off with a mix of cooperators and defectors, then there will always be fewer cooperators than optimal at steady state, even in the limit of infinitely strong competition between groups.

The replicator dynamics for multilevel selection in evolutionary games

We consider a stochastic model for evolution of group-structured populations in which interactions between group members correspond to the Prisoner's Dilemma or the Hawk-Dove game. Selection operates at two organization levels: individuals compete with peer group members based on individual payoff, while groups also compete with other groups based on average payoff of group members. In the Prisoner's Dilemma, this creates a tension between the two levels of selection, as defectors are favored at the individual level, whereas groups with at least some cooperators outperform groups of defectors at the between-group level. In the limit of infinite group size and infinite number of groups, we derive a non-local PDE that describes the probability distribution of group compositions in the population. For special families of payoff matrices, we characterize the long-time behavior of solutions of our equation, finding a threshold intensity of between-group selection required to sustain density steady states and the survival of cooperation. When all-cooperator groups are most fit, the average and most abundant group compositions at steady state range from featuring all-defector groups when individual-level selection dominates to featuring all-cooperator groups when group-level selection dominates. When the most fit groups have a mix of cooperators and defectors, then the average and most abundant group compositions always feature a smaller fraction of cooperators than required for the optimal mix, even in the limit where group-level selection is infinitely stronger than individual-level selection. In such cases, the conflict between the two levels of selection cannot be decoupled, and cooperation cannot be sustained at all in the case where between-group competition favors an even mix of cooperators and defectors.

Transient compartmentalization of RNA replicators prevents extinction due to parasites

The appearance of molecular replicators (molecules that can be copied) was probably a critical step in the origin of life. However, parasitic replicators would take over and would have prevented life from taking off unless the replicators were compartmentalized in reproducing protocells. Paradoxically, control of protocell reproduction would seem to require evolved replicators. We show here that a simpler population structure, based on cycles of transient compartmentalization (TC) and mixing of RNA replicators, is sufficient to prevent takeover by parasitic mutants. TC tends to select for ensembles of replicators that replicate at a similar rate, including a diversity of parasites that could serve as a source of opportunistic functionality. Thus, TC in natural, abiological compartments could have allowed life to take hold.

Counterintuitive properties of the fixation time in network-structured populations

Evolutionary dynamics on graphs can lead to many interesting and counterintuitive findings. We study the Moran process, a discrete time birth-death process, that describes the invasion of a mutant type into a population of wild-type individuals. Remarkably, the fixation probability of a single mutant is the same on all regular networks. But non-regular networks can increase or decrease the fixation probability. While the time until fixation formally depends on the same transition probabilities as the fixation probabilities, there is no obvious relation between them. For example, an amplifier of selection, which increases the fixation probability and thus decreases the number of mutations needed until one of them is successful, can at the same time slow down the process of fixation. Based on small networks, we show analytically that (i) the time to fixation can decrease when links are removed from the network and (ii) the node providing the best starting conditions in terms of the shortest fixation time depends on the fitness of the mutant. Our results are obtained analytically on small networks, but numerical simulations show that they are qualitatively valid even in much larger populations.

Universal scaling for the dilemma strength in evolutionary games

Why would natural selection favor the prevalence of cooperation within the groups of selfish individuals? A fruitful framework to address this question is evolutionary game theory, the essence of which is captured in the so-called social dilemmas. Such dilemmas have sparked the development of a variety of mathematical approaches to assess the conditions under which cooperation evolves. Furthermore, borrowing from statistical physics and network science, the research of the evolutionary game dynamics has been enriched with phenomena such as pattern formation, equilibrium selection, and self-organization. Numerous advances in understanding the evolution of cooperative behavior over the last few decades have recently been distilled into five reciprocity mechanisms: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity. However, when social viscosity is introduced into a population via any of the reciprocity mechanisms, the existing scaling parameters for the dilemma strength do not yield a unique answer as to how the evolutionary dynamics should unfold. Motivated by this problem, we review the developments that led to the present state of affairs, highlight the accompanying pitfalls, and propose new universal scaling parameters for the dilemma strength. We prove universality by showing that the conditions for an ESS and the expressions for the internal equilibriums in an infinite, well-mixed population subjected to any of the five reciprocity mechanisms depend only on the new scaling parameters. A similar result is shown to hold for the fixation probability of the different strategies in a finite, well-mixed population. Furthermore, by means of numerical simulations, the same scaling parameters are shown to be effective even if the evolution of cooperation is considered on the spatial networks (with the exception of highly heterogeneous setups). We close the discussion by suggesting promising directions for future research including (i) how to handle the dilemma strength in the context of co-evolution and (ii) where to seek opportunities for applying the game theoretical approach with meaningful impact.

The remarkable discreteness of being

Life is a discrete, stochastic phenomenon: for a biological organism, the time of the two most important events of its life (reproduction and death) is random and these events change the number of individuals of the species by single units. These facts can have surprising, counterintuitive consequences. I review here three examples where these facts play, or could play, important roles: the spatial distribution of species, the structuring of biodiversity and the (Darwinian) evolution of altruistic behaviour.

Continuous-time models of group selection, and the dynamical insufficiency of kin selection models

Traditionally, the process of group selection has been described mathematically by discrete-time models, and analyzed using tools like the Price equation. This approach makes implicit assumptions about the process that are not valid in general, like the central role of synchronized mass-dispersion and group re-formation events. In many important examples (like hunter-gatherer tribes) there are no mass-dispersion events, and the group-level events that do occur, like fission, fusion, and extinction, occur asynchronously. Examples like these can be fully analyzed by the equations of two-level population dynamics (described here) so their models are dynamically sufficient. However, it will be shown that examples like these cannot be fully analyzed by kin selection (inclusive fitness) methods because kin selection versions of group selection models are not dynamically sufficient. This is a critical mathematical difference between group selection and kin selection models, which implies that the two theories are not mathematically equivalent.

Heterogeneity in background fitness acts as a suppressor of selection

We introduce the concept of heterogeneity in background fitness to evolutionary dynamics in finite populations. Background fitness is specific to an individual but not linked to its strategy. It can be thought of as a property that is related to the physical or societal position of an individual, but is not dependent on the strategy that is adopted in the evolutionary process under consideration. In our model, an individual's total fitness is the sum of its background fitness and the fitness derived from using a specific strategy. This approach has important implications for the imitation of behavioural strategies: if we imitate others for their success, but can only adopt their behaviour and not their social and economic ties, we may imitate in vain. We study the effect of heterogeneity in background fitness on the fixation of a mutant strategy with constant fitness. We find that heterogeneity suppresses selection, but also decreases the time until a novel strategy either takes over the population or is lost again. We derive analytical solutions of the fixation probability in small populations. In the case of large total background fitness in a population with maximum inequality, we find a particularly simple approximation of the fixation probability. Numerical simulations suggest that this simple approximation also holds for larger population sizes.

The effect of population structure on the rate of evolution

Ecological factors exert a range of effects on the dynamics of the evolutionary process. A particularly marked effect comes from population structure, which can affect the probability that new mutations reach fixation. Our interest is in population structures, such as those depicted by 'star graphs', that amplify the effects of selection by further increasing the fixation probability of advantageous mutants and decreasing the fixation probability of disadvantageous mutants. The fact that star graphs increase the fixation probability of beneficial mutations has lead to the conclusion that evolution proceeds more rapidly in star-structured populations, compared with mixed (unstructured) populations. Here, we show that the effects of population structure on the rate of evolution are more complex and subtle than previously recognized and draw attention to the importance of fixation time. By comparing population structures that amplify selection with other population structures, both analytically and numerically, we show that evolution can slow down substantially even in populations where selection is amplified.

Selfishness versus functional cooperation in a stochastic protocell model

How to design an "evolvable" artificial system capable to increase in complexity? Although Darwin's theory of evolution by natural selection obviously offers a firm foundation, little hope of success seems to be expected from the explanatory adequacy of modern evolutionary theory, which does a good job at explaining what has already happened but remains practically helpless at predicting what will occur. However, the study of the major transitions in evolution clearly suggests that increases in complexity have occurred on those occasions when the conflicting interests between competing individuals were partly subjugated. This immediately raises the issue about "levels of selection" in evolutionary biology, and the idea that multi-level selection scenarios are required for complexity to emerge. After analyzing the dynamical behaviour of competing replicators within compartments, we show here that a proliferation of differentiated catalysts and/or improvement of catalytic efficiency of ribozymes can potentially evolve in properly designed artificial cells where the strong internal competition between the different species of replicators is somewhat prevented (i.e., by choosing them with equal probability). Experimental evolution in these systems will likely stand as beautiful examples of artificial adaptive systems, and will provide new insights to understand possible evolutionary paths to the evolution of metabolic complexity.

Analytical results for individual and group selection of any intensity

The idea of evolutionary game theory is to relate the payoff of a game to reproductive success (= fitness). An underlying assumption in most models is that fitness is a linear function of the payoff. For stochastic evolutionary dynamics in finite populations, this leads to analytical results in the limit of weak selection, where the game has a small effect on overall fitness. But this linear function makes the analysis of strong selection difficult. Here, we show that analytical results can be obtained for any intensity of selection, if fitness is defined as an exponential function of payoff. This approach also works for group selection (= multi-level selection). We discuss the difference between our approach and that of inclusive fitness theory.

The analysis of bi-level evolutionary graphs

Evolutionary graphs (EGs), in which evolutionary dynamic is arranged on a graph, were initially proposed by Lieberman et al. [Lieberman, E., Hauert, C., Nowak, M.A., 2005. Evolutionary dynamics on graphs. Nature 433, 312-316] in the biological field and many biological phenomena are successfully explained. EGs on two levels (or bi-level EGs), based on some biological phenomena, are considered in this paper. The bi-level EGs are compared with the one-rooted EGs in two cases. One has the identical numbers of the followers, the other with the same numbers of total individuals. Then, some properties of the bi-level EGs are obtained. It is showed that bi-level EGs are more stable, and the bi-level EGs with just two leaders are the most stable, if they have identical followers respectively. The bi-level EGs theory can successfully explain the phenomena of symbiosis in biology.

Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S. In a well-mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.

The evolution of norms

We develop a two-level selection model in the framework of evolutionary game theory, in which fitness selection at different levels is related to different games. We consider an archipelago of communities, such that selection operates at an individual level inside each community and at a group level whenever evolution of communities is at stake. We apply this model to the evolution of social norms, an open problem of ubiquitous importance in social science. Extensive statistical analysis of our results lead to the emergence of one common social norm, of which the evolutionary outcomes in different communities are simple by-products. This social norm induces reputation-based cooperative behavior, and reflects the evolutionary propensity to promote simple, unambiguous norms, in which forgiveness and repent are welcome, while punishment is implacable.