Extrapolating weak selection in evolutionary games

Explosive Cooperation in Social Dilemmas on Higher-Order Networks

Understanding how cooperative behaviors can emerge from competitive interactions is an open problem in biology and social sciences. While interactions are usually modeled as pairwise networks, the units of many real-world systems can also interact in groups of three or more. Here, we introduce a general framework to extend pairwise games to higher-order networks. By studying social dilemmas on hypergraphs with a tunable structure, we find an explosive transition to cooperation triggered by a critical number of higher-order games. The associated bistable regime implies that an initial critical mass of cooperators is also required for the emergence of prosocial behavior. Our results show that higher-order interactions provide a novel explanation for the survival of cooperation.

Strategy evolution on higher-order networks

Cooperation is key to prosperity in human societies. Population structure is well understood as a catalyst for cooperation, where research has focused on pairwise interactions. But cooperative behaviors are not simply dyadic, and they often involve coordinated behavior in larger groups. Here we develop a framework to study the evolution of behavioral strategies in higher-order population structures, which include pairwise and multi-way interactions. We provide an analytical treatment of when cooperation will be favored by higher-order interactions, accounting for arbitrary spatial heterogeneity and nonlinear rewards for cooperation in larger groups. Our results indicate that higher-order interactions can act to promote the evolution of cooperation across a broad range of networks, in public goods games. Higher-order interactions consistently provide an advantage for cooperation when interaction hyper-networks feature multiple conjoined communities. Our analysis provides a systematic account of how higher-order interactions modulate the evolution of prosocial traits.

Cooperation among unequal players with aspiration-driven learning

Direct reciprocity promotes the evolution of cooperation when players are sufficiently equal, such that they have similar influence on each other. In the light of ubiquitous inequality, this raises the question of how reciprocity evolves among unequal players. Existing studies on inequality mainly focus on payoff-driven learning rules, which rely on the knowledge of others' strategies. However, inferring one's strategy is a difficult task even if the whole interaction history is known. Here, we consider aspiration-driven learning rules, where players seek strategies that satisfy their aspirations based on their own information. Under aspiration-driven learning rules, we explore the evolutionary dynamics among players with inequality in endowments and productivity. We model the interactions among unequal players with asymmetric games and characterize the condition where cooperation is feasible. Remarkably, we find that aspiration-driven learning rules lead to a higher level of cooperation than payoff-driven ones over a wide range of inequality. Moreover, our results show that high aspiration levels are conducive to the evolution of cooperation when more productive players are equipped with higher endowments. Our work highlights the advantages of aspiration-driven learning for promoting cooperation among unequal players and suggests that aspiration-based decision-making may be more beneficial for the collective.

Weak selection and stochastic evolutionary stability in a stochastic replicator dynamics

It is a very challenging problem whether natural selection is able to effectively resist the continuous disturbance of environmental noise such that the direction or outcome of evolution determined by the deterministic selection pressure will not be changed. By analyzing the impact of weak selection on the evolutionary stability of a stochastic replicator dynamics with n possible pure strategies, we found that the weak selection is able to enhance the evolutionary stability, that is, under weak selection, the stochastic evolutionary stability of the system is determined by the mean payoff matrix. This finding strongly implies that the weak selection should be regarded as an important mechanism to ensure evolutionary stability in stochastic environments.

Evolution of cooperation with joint liability

'Personal responsibility', one of the basic principles of social governance, requires one to be accountable for what one does. However, personal responsibility is far from the only norm ruling human interactions, especially in social and economic activities. In many collective communities such as among enterprise colleagues and family members, one's personal interests are often bound to others'-once one member breaks the rule, a group of people have to bear the punishment or sanction. Such a mechanism is termed 'joint liability'. Although many real-world cases have evidenced that joint liability can help to maintain collective collaboration, a deep and systematic theoretical analysis on how and when it promotes cooperation remains lacking. Here, we use evolutionary game theory to model an interacting system with joint liability, where one's losing credit could deteriorate the reputation of the whole group. We provide the analytical condition to predict when cooperation evolves and analytically prove that in the presence of punishment, being jointly liable greatly promotes cooperation. Our work stresses that joint liability is of great significance in promoting current economic prosperity.

Weak Selection and the Separation of Eco-evo Time Scales using Perturbation Analysis

We show that under the assumption of weak frequency-dependent selection a wide class of population dynamical models can be analysed using perturbation theory. The inner solution corresponds to the ecological dynamics, where to zeroth order, the genotype frequencies remain constant. The outer solution provides the evolutionary dynamics and corresponds, to zeroth order, to a generalisation of the replicator equation. We apply this method to a model of public goods dynamics and construct, using matched asymptotic expansions, a composite solution valid for all times. We also analyse a Lotka-Volterra model of predator competition and show that to zeroth order the fraction of wild-type predators follows a replicator equation with a constant selection coefficient given by the predator death rate. For both models, we investigate how the error between approximate solutions and the solution to the full model depend on the order of the approximation and show using numerical comparison, for [Formula: see text] and 2, that the error scales according to [Formula: see text], where [Formula: see text] is the strength of selection and k is the order of the approximation.

Evolution of prosocial behaviours in multilayer populations

Human societies include diverse social relationships. Friends, family, business colleagues and online contacts can all contribute to one's social life. Individuals may behave differently in different domains, but success in one domain may engender success in another. Here, we study this problem using multilayer networks to model multiple domains of social interactions, in which individuals experience different environments and may express different behaviours. We provide a mathematical analysis and find that coupling between layers tends to promote prosocial behaviour. Even if prosociality is disfavoured in each layer alone, multilayer coupling can promote its proliferation in all layers simultaneously. We apply this analysis to six real-world multilayer networks, ranging from the socio-emotional and professional relationships in a Zambian community, to the online and offline relationships within an academic university. We discuss the implications of our results, which suggest that small modifications to interactions in one domain may catalyse prosociality in a different domain.

Weak selection helps cheap but harms expensive cooperation in spatial threshold dilemmas

Public Goods Games (PGGs) are n-person games with dependence of individual fitness benefits on the collective investment by the players. We have studied a simple PGG scenario played out by cooperating (C) and defecting (D) agents, applying the highly nonlinear threshold benefit function in an individual-based lattice model. A semi-analytical approximation of the lattice model has been developed and shown to describe the dynamics fairly well in the vicinity of the steady state. Besides the expected outcomes (i.e., the negative effect on cooperator persistence of higher cooperation costs and/or more intensive mixing of the population) we have found a surprising, counter-intuitive effect of the strength of selection on the steady state of the model. The effect is different at low and high cooperation costs, and it shows up only in the lattice model, suggesting that stochastic effects and higher order spatial correlations due to the emergent spatial clustering of cooperators (not taken into account in the semi-analytical approximation) must be responsible for the unexpected results for which we propose an intuitive explanation, present a tentative demonstration, and shortly discuss their biological relevance.

Intriguing effects of selection intensity on the evolution of prosocial behaviors

In many models of evolving populations, genetic drift has an outsized role relative to natural selection, or vice versa. While there are many scenarios in which one of these two assumptions is reasonable, intermediate balances between these forces are also biologically relevant. In this study, we consider some natural axioms for modeling intermediate selection intensities, and we explore how to quantify the long-term evolutionary dynamics of such a process. To illustrate the sensitivity of evolutionary dynamics to drift and selection, we show that there can be a “sweet spot” for the balance of these two forces, with sufficient noise for rare mutants to become established and sufficient selection to spread. This balance allows prosocial traits to evolve in evolutionary models that were previously thought to be unconducive to the emergence and spread of altruistic behaviors. Furthermore, the effects of selection intensity on long-run evolutionary outcomes in these settings, such as when there is global competition for reproduction, can be highly non-monotonic. Although intermediate selection intensities (neither weak nor strong) are notoriously difficult to study analytically, they are often biologically relevant; and the results we report suggest that they can elicit novel and rich dynamics in the evolution of prosocial behaviors.

Theoretical models of populations have been useful for assessing when and how traits spread, in large part because they are simple. Rather than being used to reproduce empirical data, these idealized models involve relatively few parameters and are utilized to gain a qualitative understanding of what promotes or suppresses a trait. For prosocial traits, which entail a cost to self to help another, one thing that mathematical models often suggest is that competition to reproduce must be localized, meaning an individual must be fitter than just a small subset of the population in order to produce an offspring. We show here that this finding is not robust. Such traits can indeed proliferate when there is global competition for reproduction, which we demonstrate by increasing the degree to which payoffs from games affect birth rates. Since this kind of “stronger selection” has also been observed empirically, we discuss how it is incorporated into theoretical models more broadly.

Evolution of state-dependent strategies in stochastic games

In a population of interacting individuals, the environment for interactions often changes due to individuals' behaviors, which in turn drive the evolution of individuals' behaviors. The interplay between the environment and individuals' behaviors has been demonstrated to remarkably influence the evolutionary outcomes. In reality, in highly cognitive species such as social primates and human beings, individuals are often capable of perceiving the environment change and then differentiate their strategies across different environment states. We propose a model of environmental feedback with state-dependent strategies: individuals have perceptions of distinct environment states and therefore take distinct sub-strategies under each of them; based on the sub-strategy, individuals then decide their behaviors; their behaviors subsequently modify the environment state. We use the theory of stochastic games and evolutionary dynamics to analyze this idea. We find that when environment changes slower than behaviors, state-dependent strategies (i.e. taking different sub-strategies under different environment states) can outperform state-independent strategies (i.e. taking an identical sub-strategy under all environment states), such as Win-Stay, Lose-Shift, the most leading strategy among state-independent strategies. The intuition is that delayed environmental feedback provides chances for individuals with state-dependent strategies to exploit those with state-independent strategies. Our results hold (1) in both well-mixed and structured populations; (2) when the environment switches between two or more states. Furthermore, the environment changing rate decides if state-dependent strategies benefit global cooperation. The evolution sees the rise of the cooperation level for fast environment switching and the decrease otherwise. Our work stresses that individuals' perceptions of different environment states are beneficial to their survival and social prosperity in a changing world.

Aspiration dynamics generate robust predictions in heterogeneous populations

Update rules, which describe how individuals adjust their behavior over time, affect the outcome of social interactions. Theoretical studies have shown that evolutionary outcomes are sensitive to model details when update rules are imitation-based but are robust when update rules are self-evaluation based. However, studies of self-evaluation based rules have focused on homogeneous population structures where each individual has the same number of neighbors. Here, we consider heterogeneous population structures represented by weighted networks. Under weak selection, we analytically derive the condition for strategy success, which coincides with the classical condition of risk-dominance. This condition holds for all weighted networks and distributions of aspiration levels, and for individualized ways of self-evaluation. Our findings recover previous results as special cases and demonstrate the universality of the robustness property under self-evaluation based rules. Our work thus sheds light on the intrinsic difference between evolutionary dynamics under self-evaluation based and imitation-based update rules.

Social interaction outcomes can depend on the type of information individuals possess and how it is used in decision-making. Here, Zhou et al. find that self-evaluation based decision-making rules lead to evolutionary outcomes that are robust to different population structures and ways of self-evaluation.

Evolution of cooperation in social dilemmas under the coexistence of aspiration and imitation mechanisms

Imitation and aspiration update rules are frequently observed in human and animal populations. While the imitation process entails payoff comparisons with surroundings, the aspiration process refers to self-evaluation. This work explores the evolution of cooperation in dyadic games under the coexistence of these two dynamics in an infinitely large well-mixed population. Two situations have been explored: (i) individuals adopt either an imitation or aspiration update rule with a certain probability, and (ii) the entire population is divided into two groups where one group only uses imitative rules and the other obeys aspiration updating alone. Both premises have been modeled by taking an infinite approximation of the finite population. In particular, the second mixing principle follows an additive property: the outcome of the whole population is the weighted average of outcomes from imitators and aspiration-driven individuals. Our work progressively investigates several variants of aspiration dynamics under strong selection, encompassing symmetric, asymmetric, and adaptive aspirations, which then coalesce with imitative dynamics. We also demonstrate which of the update rules performs better, under different social dilemmas, by allowing the evolution of the preference of update rules besides strategies. Aspiration dynamics always outperform imitation dynamics in the prisoner's dilemma, however, in the chicken and stag-hunt games the predominance of either update rule depends on the level of aspirations as well as on the extent of greed and fear present in the system. Finally, we examine the coevolution of strategies, aspirations, and update rules which leads to a binary state of obeying either imitation or aspiration dynamics. In such a circumstance, when aspiration dynamics prevail over imitation dynamics, cooperators and defectors coexist to an equal extent.

The cancellation effect at the group level

Group selection models combine selection pressure at the individual level with selection pressure at the group level. Cooperation can be costly for individuals, but beneficial for the group, and therefore, if individuals are sufficiently much assorted, and cooperators find themselves in groups with disproportionately many other cooperators, cooperation can evolve. The existing literature on group selection generally assumes that competition between groups takes place in a well-mixed population of groups, where any group competes with any other group equally intensely. Competition between groups however might very well occur locally; groups may compete more intensely with nearby than with far-away groups. We show that if competition between groups is indeed local, then the evolution of cooperation can be hindered significantly by the fact that groups with many cooperators will mostly compete against neighboring groups that are also highly cooperative, and therefore harder to outcompete. The existing empirical method for determining how conducive a group structured population is to the evolution of cooperation also implicitly assumes global between-group competition, and therefore gives (possibly very) biased estimates.

Weak selection can filter environmental noise in the evolution of animal behavior

Weak selection is an important assumption in theoretical evolutionary biology, but its biological significance remains unclear. In this study, we investigate the effect of weak selection on stochastic evolutionary stability in a two-phenotype evolutionary game dynamics with a random payoff matrix assuming an infinite, well-mixed population undergoing discrete, nonoverlapping generations. We show that, under weak selection, both stochastic local stability and stochastic evolutionary stability in this system depend on the means of the random payoffs but not on their variances. Moreover, although stochastic local stability or instability of an equilibrium may not depend on environmental noise if selection is weak enough, the growth rate near an equilibrium not only depends on environmental noise, but can even be enhanced by environmental noise if selection is weak. This is the case, for instance, when the variances of the random payoffs as well as the covariances are equal. These results suggest that natural selection could be able to filter (or resist) the effect of environmental noise on the evolution of animal behavior if selection is weak.

Evolutionary dynamics with game transitions

The environment has a strong influence on a population's evolutionary dynamics. Driven by both intrinsic and external factors, the environment is subject to continual change in nature. To capture an ever-changing environment, we consider a model of evolutionary dynamics with game transitions, where individuals' behaviors together with the games that they play in one time step influence the games to be played in the next time step. Within this model, we study the evolution of cooperation in structured populations and find a simple rule: Weak selection favors cooperation over defection if the ratio of the benefit provided by an altruistic behavior, b, to the corresponding cost, c, exceeds [Formula: see text], where k is the average number of neighbors of an individual and [Formula: see text] captures the effects of the game transitions. Even if cooperation cannot be favored in each individual game, allowing for a transition to a relatively valuable game after mutual cooperation and to a less valuable game after defection can result in a favorable outcome for cooperation. In particular, small variations in different games being played can promote cooperation markedly. Our results suggest that simple game transitions can serve as a mechanism for supporting prosocial behaviors in highly connected populations.

Computation and Simulation of Evolutionary Game Dynamics in Finite Populations

The study of evolutionary dynamics increasingly relies on computational methods, as more and more cases outside the range of analytical tractability are explored. The computational methods for simulation and numerical approximation of the relevant quantities are diverging without being compared for accuracy and performance. We thoroughly investigate these algorithms in order to propose a reliable standard. For expositional clarity we focus on symmetric 2 × 2 games leading to one-dimensional processes, noting that extensions can be straightforward and lessons will often carry over to more complex cases. We provide time-complexity analysis and systematically compare three families of methods to compute fixation probabilities, fixation times and long-term stationary distributions for the popular Moran process. We provide efficient implementations that substantially improve wall times over naive or immediate implementations. Implications are also discussed for the Wright-Fisher process, as well as structured populations and multiple types.

Properties of network structures, structure coefficients, and benefit-to-cost ratios

In structured populations the spatial arrangement of cooperators and defectors on the interaction graph together with the structure of the graph itself determines the game dynamics and particularly whether or not fixation of cooperation (or defection) is favored. For networks described by regular graphs and for a single cooperator (and a single defector) the question of fixation can be addressed by a single parameter, the structure coefficient. This quantity is invariant with respect to the location of the cooperator on the graph and also does not vary over different networks. We may therefore consider it to be generic for regular graphs and call it the generic structure coefficient. For two and more cooperators (or several defectors) fixation properties can also be assigned by structure coefficients. These structure coefficients, however, depend on the arrangement of cooperators and defectors which we may interpret as a configuration of the game. Moreover, the coefficients are specific for a given interaction network modeled as a regular graph, which is why we may call them specific structure coefficients. In this paper, we study how specific structure coefficients vary over interaction graphs and analyze how spectral properties of interaction networks relate to specific structure coefficients. We also discuss implications for the benefit-to-cost ratios of donation games.

Fixation properties of multiple cooperator configurations on regular graphs

Whether or not cooperation is favored in evolutionary games on graphs depends on the population structure and spatial properties of the interaction network. The population structure can be expressed as configurations. Such configurations extend scenarios with a single cooperator among defectors to any number of cooperators and any arrangement of cooperators and defectors on the network. For interaction networks modeled as regular graphs and for weak selection, the emergence of cooperation can be assessed by structure coefficients, which can be specified for each configuration and each regular graph. Thus, as a single cooperator can be interpreted as a lone mutant, the configuration-based structure coefficients also describe fixation properties of multiple mutants. We analyze the structure coefficients and particularly show that under certain conditions, the coefficients strongly correlate to the average shortest path length between cooperators on the evolutionary graph. Thus, for multiple cooperators fixation properties on regular evolutionary graphs can be linked to cooperator path lengths.

Evolution of Groupwise Cooperation: Generosity, Paradoxical Behavior, and Non-Linear Payoff Functions

Evolution of cooperation by reciprocity has been studied using two-player and n-player repeated prisoner’s dilemma games. An interesting feature specific to the n-player case is that players can vary in generosity, or how many defections they tolerate in a given round of a repeated game. Reciprocators are quicker to detect defectors to withdraw further cooperation when less generous, and better at maintaining a long-term cooperation in the presence of rare defectors when more generous. A previous analysis on a stochastic evolutionary model of the n-player repeated prisoner’s dilemma has shown that the fixation probability of a single reciprocator in a population of defectors can be maximized for a moderate level of generosity. However, the analysis is limited in that it considers only tit-for-tat-type reciprocators within the conventional linear payoff assumption. Here we extend the previous study by removing these limitations and show that, if the games are repeated sufficiently many times, considering non-tit-for-tat type strategies does not alter the previous results, while the introduction of non-linear payoffs sometimes does. In particular, under certain conditions, the fixation probability is maximized for a “paradoxical” strategy, which cooperates in the presence of fewer cooperating opponents than in other situations in which it defects.

Stationary frequencies and mixing times for neutral drift processes with spatial structure

We study a general setting of neutral evolution in which the population is of finite, constant size and can have spatial structure. Mutation leads to different genetic types (traits), which can be discrete or continuous. Under minimal assumptions, we show that the marginal trait distributions of the evolutionary process, which specify the probability that any given individual has a certain trait, all converge to the stationary distribution of the mutation process. In particular, the stationary frequencies of traits in the population are independent of its size, spatial structure and evolutionary update rule, and these frequencies can be calculated by evaluating a simple stochastic process describing a population of size one (i.e. the mutation process itself). We conclude by analysing mixing times, which characterize rates of convergence of the mutation process along the lineages, in terms of demographic variables of the evolutionary process.

Individualised aspiration dynamics: Calculation by proofs

Cooperation is key for the evolution of biological systems ranging from bacteria communities to human societies. Evolutionary processes can dramatically alter the cooperation level. Evolutionary processes are typically of two classes: comparison based and self-evaluation based. The fate of cooperation is extremely sensitive to the details of comparison based processes. For self-evaluation processes, however, it is still unclear whether the sensitivity remains. We concentrate on a class of self-evaluation processes based on aspiration, where all the individuals adjust behaviors based on their own aspirations. We prove that the evolutionary outcome with heterogeneous aspirations is the same as that of the homogeneous one for regular networks under weak selection limit. Simulation results further suggest that it is also valid for general networks across various distributions of personalised aspirations. Our result clearly indicates that self-evaluation processes are robust in contrast with comparison based rules. In addition, our result greatly simplifies the calculation of the aspiration dynamics, which is computationally expensive.

Cooperation is the cornerstone to understand how biological systems evolve. Previous studies have shown that cooperation is sensitive to the details of evolutionary processes, even if all the individuals update strategies in the same way. Here we propose a class of updating rules driven by self-evaluation, where each individual has its personal aspiration. The evolutionary outcome is the same as if all the individuals adopt the same aspiration for regular networks, provided the selection intensity is weak enough. In addition, we provide a simple numerical method to identify the favored strategy. Our result shows a very robust class of strategy updating rules. And it implies that complexity in updating rules does not necessarily lead to the sensitivity of evolutionary outcomes.

Extrapolating weak selection in evolutionary games

This work is inspired by a 2013 paper from Arne Traulsen's lab at the Max Plank Institute for Evolutionary Biology (Wu et al. in PLoS Comput Biol 9:e1003381, 2013). They studied evolutionary games when the mutation rate is so small that each mutation goes to fixation before the next one occurs. It has been shown that for [Formula: see text] games the ranking of the strategies does not change as strength of selection is increased (Wu et al. in Phys Rev 82:046106, 2010). The point of the 2013 paper is that when there are three or more strategies the ordering can change as selection is increased. Wu et al. (2013) did numerical computations for a fixed population size N. Here, we will instead let the strength of selection [Formula: see text] where c is fixed and let [Formula: see text] to obtain formulas for the invadability probabilities [Formula: see text] that determine the rankings. These formulas, which are integrals on [0, 1], are intractable calculus problems, but can be easily evaluated numerically. Here, we use them to derive simple formulas for the ranking order when c is small or c is large.

Public goods games in populations with fluctuating size

Many mathematical frameworks of evolutionary game dynamics assume that the total population size is constant and that selection affects only the relative frequency of strategies. Here, we consider evolutionary game dynamics in an extended Wright-Fisher process with variable population size. In such a scenario, it is possible that the entire population becomes extinct. Survival of the population may depend on which strategy prevails in the game dynamics. Studying cooperative dilemmas, it is a natural feature of such a model that cooperators enable survival, while defectors drive extinction. Although defectors are favored for any mixed population, random drift could lead to their elimination and the resulting pure-cooperator population could survive. On the other hand, if the defectors remain, then the population will quickly go extinct because the frequency of cooperators steadily declines and defectors alone cannot survive. In a mutation-selection model, we find that (i) a steady supply of cooperators can enable long-term population survival, provided selection is sufficiently strong, and (ii) selection can increase the abundance of cooperators but reduce their relative frequency. Thus, evolutionary game dynamics in populations with variable size generate a multifaceted notion of what constitutes a trait's long-term success.

Evolving synergetic interactions

Cooperators forgo their own interests to benefit others. This reduces their fitness and thus cooperators are not likely to spread based on natural selection. Nonetheless, cooperation is widespread on every level of biological organization ranging from bacterial communities to human society. Mathematical models can help to explain under which circumstances cooperation evolves. Evolutionary game theory is a powerful mathematical tool to depict the interactions between cooperators and defectors. Classical models typically involve either pairwise interactions between individuals or a linear superposition of these interactions. For interactions within groups, however, synergetic effects may arise: their outcome is not just the sum of its parts. This is because the payoffs via a single group interaction can be different from the sum of any collection of two-player interactions. Assuming that all interactions start from pairs, how can such synergetic multiplayer games emerge from simpler pairwise interactions? Here, we present a mathematical model that captures the transition from pairwise interactions to synergetic multiplayer ones. We assume that different social groups have different breaking rates. We show that non-uniform breaking rates do foster the emergence of synergy, even though individuals always interact in pairs. Our work sheds new light on the mechanisms underlying such synergetic interactions.

Which facilitates the evolution of cooperation more, retaliation or persistence?

The existence of cooperation in this world is a mysterious phenomenon. One of the mechanisms that explain the evolution of cooperation is repeated interaction. If interactions between the same individuals repeat and individuals cooperate conditionally, cooperation can evolve. A previous study pointed out that if individuals have persistence (i.e., imitate its "own" behavior in the last move), cooperation can evolve. However, retaliation and persistence are not mutually exclusive decisions, but rather a trade-off in the decision making process of individuals. Players can refer to the opponent's behavior and if the actor and the opponent opted for the different alternative in the last move, conditional cooperators have to give up either retaliation or persistence. The previous study also investigated this, and has revealed that the individual should give more importance to retaliation than to persistence. However, this study has assumed that the errors in perception are absent. In this world, errors in perception are present, and trying to imitate the opponent player can sometimes end in failure. And, it might be that imitating the focal player, which definitely ends in success, is more beneficial than trying to imitate the opponent player, which can end in failure especially when the error rate in recognition is large. Here, this paper uses evolutionarily stable strategy (ESS) analysis and analyzes the stability for reactive strategies against the invasion by unconditional defectors in the iterated prisoner's dilemma game. And our analysis reveals that even if we take errors in perception into consideration, retaliation facilitates the evolution of cooperation more than persistence unexpectedly. In addition, we analyze the stability for reactive cooperators against the invasion by a strategy other than unconditional defectors. Moreover, we also analyze the deterministic model in which unconditional cooperators, unconditional defectors, and the reactive strategy at the same time.

Extreme Selection Unifies Evolutionary Game Dynamics in Finite and Infinite Populations

We show that when selection is extreme-the fittest strategy always reproduces or is imitated-the unequivalence between the possible evolutionary game scenarios in finite and infinite populations resolves, in the sense that the three generic outcomes-dominance, coexistence, and mutual exclusion-emerge in well-mixed populations of any size. We consider the simplest setting of a 2-player-2-strategy symmetric game and the two most common microscopic definitions of strategy spreading-the frequency-dependent Moran process and the imitation process by pairwise comparison-both in the case allowing any intensity of selection. We show that of the seven different invasion and fixation scenarios that are generically possible in finite populations-fixation being more or less likely to occur and rapid compared to the neutral game-the three that are possible in large populations are the same three that occur for sufficiently strong selection: (1) invasion and fast fixation of one strategy; (2) mutual invasion and slow fixation of one strategy; (3) no invasion and no fixation. Moreover (and interestingly), in the limit of extreme selection 2 becomes mutual invasion and no fixation, a case not possible for finite intensity of selection that better corresponds to the deterministic case of coexistence. In the extreme selection limit, we also derive the large population deterministic limit of the two considered stochastic processes.

Evolutionary games on cycles with strong selection

Evolutionary games on graphs describe how strategic interactions and population structure determine evolutionary success, quantified by the probability that a single mutant takes over a population. Graph structures, compared to the well-mixed case, can act as amplifiers or suppressors of selection by increasing or decreasing the fixation probability of a beneficial mutant. Properties of the associated mean fixation times can be more intricate, especially when selection is strong. The intuition is that fixation of a beneficial mutant happens fast in a dominance game, that fixation takes very long in a coexistence game, and that strong selection eliminates demographic noise. Here we show that these intuitions can be misleading in structured populations. We analyze mean fixation times on the cycle graph under strong frequency-dependent selection for two different microscopic evolutionary update rules (death-birth and birth-death). We establish exact analytical results for fixation times under strong selection and show that there are coexistence games in which fixation occurs in time polynomial in population size. Depending on the underlying game, we observe inherence of demographic noise even under strong selection if the process is driven by random death before selection for birth of an offspring (death-birth update). In contrast, if selection for an offspring occurs before random removal (birth-death update), then strong selection can remove demographic noise almost entirely.

Coevolutionary dynamics of phenotypic diversity and contingent cooperation

Phenotypic diversity is considered beneficial to the evolution of contingent cooperation, in which cooperators channel their help preferentially towards others of similar phenotypes. However, it remains largely unclear how phenotypic variation arises in the first place and thus leads to the construction of phenotypic complexity. Here we propose a mathematical model to study the coevolutionary dynamics of phenotypic diversity and contingent cooperation. Unlike previous models, our model does not assume any prescribed level of phenotypic diversity, but rather lets it be an evolvable trait. Each individual expresses one phenotype at a time and only the phenotypes expressed are visible to others. Moreover, individuals can differ in their potential of phenotypic variation, which is characterized by the number of distinct phenotypes they can randomly switch to. Each individual incurs a cost proportional to the number of potentially expressible phenotypes so as to retain phenotypic variation and expression. Our results show that phenotypic diversity coevolves with contingent cooperation under a wide range of conditions and that there exists an optimal level of phenotypic diversity best promoting contingent cooperation. It pays for contingent cooperators to elevate their potential of phenotypic variation, thereby increasing their opportunities of establishing cooperation via novel phenotypes, as these new phenotypes serve as secret tags that are difficult for defector to discover and chase after. We also find that evolved high levels of phenotypic diversity can occasionally collapse due to the invasion of defector mutants, suggesting that cooperation and phenotypic diversity can mutually reinforce each other. Thus, our results provide new insights into better understanding the coevolution of cooperation and phenotypic diversity.

Phenotypic variation is commonly observed from human cells to the intestinal pathogen Salmonella enterica serovar Typhimurium to the wrinkly-spreader morphs. Such phenotypic diversity proves effective in promoting cooperation, or confers survival advantage against unfavorable environmental changes. Prior studies show that interactions based on phenotypic similarity can promote cooperation. Yet in these models, the level of phenotypic diversity is prescribed such that individuals each possess the same number of available phenotypes, and thereby no evolution of phenotypic diversity per se. We here take into consideration important aspects of the diversity of phenotype and contingent cooperation and show that phenotypic diversity coevolves with cooperation under a variety of conditions. Our work provides a potential mechanism for the evolution of cooperation, and individuals, especially cooperators, endowed with diverse phenotypes constitute the backbone in inducing the coevolution.

Fixation Probabilities for Any Configuration of Two Strategies on Regular Graphs

Population structure and spatial heterogeneity are integral components of evolutionary dynamics, in general, and of evolution of cooperation, in particular. Structure can promote the emergence of cooperation in some populations and suppress it in others. Here, we provide results for weak selection to favor cooperation on regular graphs for any configuration, meaning any arrangement of cooperators and defectors. Our results extend previous work on fixation probabilities of rare mutants. We find that for any configuration cooperation is never favored for birth-death (BD) updating. In contrast, for death-birth (DB) updating, we derive a simple, computationally tractable formula for weak selection to favor cooperation when starting from any configuration containing any number of cooperators. This formula elucidates two important features: (i) the takeover of cooperation can be enhanced by the strategic placement of cooperators and (ii) adding more cooperators to a configuration can sometimes suppress the evolution of cooperation. These findings give a formal account for how selection acts on all transient states that appear in evolutionary trajectories. They also inform the strategic design of initial states in social networks to maximally promote cooperation. We also derive general results that characterize the interaction of any two strategies, not only cooperation and defection.

Impact of migration on the multi-strategy selection in finite group-structured populations

For large quantities of spatial models, the multi-strategy selection under weak selection is the sum of two competition terms: the pairwise competition and the competition of multiple strategies with equal frequency. Two parameters σ₁ and σ₂ quantify the dependence of the multi-strategy selection on these two terms, respectively. Unlike previous studies, we here do not require large populations for calculating σ₁ and σ₂, and perform the first quantitative analysis of the effect of migration on them in group-structured populations of any finite sizes. The Moran and the Wright-Fisher process have the following common findings. Compared with well-mixed populations, migration causes σ₁ to change with the mutation probability from a decreasing curve to an inverted U-shaped curve and maintains the increase of σ₂. Migration (probability and range) leads to a significant change of σ₁ but a negligible one of σ₂. The way that migration changes σ₁ is qualitatively similar to its influence on the single parameter characterizing the two-strategy selection. The Moran process is more effective in increasing σ₁ for most migration probabilities and the Wright-Fisher process is always more effective in increasing σ₂. Finally, our findings are used to study the evolution of cooperation under direct reciprocity.

Reputation-Based Investment Helps to Optimize Group Behaviors in Spatial Lattice Networks

Encouraging cooperation among selfish individuals is crucial in many real-world systems, where individuals’ collective behaviors can be analyzed using evolutionary public goods game. Along this line, extensive studies have shown that reputation is an effective mechanism to investigate the evolution of cooperation. In most existing studies, participating individuals in a public goods game are assumed to contribute unconditionally into the public pool, or they can choose partners based on a common reputation standard (e.g., preferences or characters). However, to assign one reputation standard for all individuals is impractical in many real-world deployment. In this paper, we introduce a reputation tolerance mechanism that allows an individual to select its potential partners and decide whether or not to contribute an investment to the public pool based on its tolerance to other individuals’ reputation. Specifically, an individual takes part in a public goods game only if the number of participants with higher reputation exceeds the value of its tolerance. Moreover, in this paper, an individual’s reputation can increase or decrease in a bounded interval based on its historical behaviors. We explore the principle that how the reputation tolerance and conditional investment mechanisms can affect the evolution of cooperation in spatial lattice networks. Our simulation results demonstrate that a larger tolerance value can achieve an environment that promote the cooperation of participants.

Asymmetric Evolutionary Games

Evolutionary game theory is a powerful framework for studying evolution in populations of interacting individuals. A common assumption in evolutionary game theory is that interactions are symmetric, which means that the players are distinguished by only their strategies. In nature, however, the microscopic interactions between players are nearly always asymmetric due to environmental effects, differing baseline characteristics, and other possible sources of heterogeneity. To model these phenomena, we introduce into evolutionary game theory two broad classes of asymmetric interactions: ecological and genotypic. Ecological asymmetry results from variation in the environments of the players, while genotypic asymmetry is a consequence of the players having differing baseline genotypes. We develop a theory of these forms of asymmetry for games in structured populations and use the classical social dilemmas, the Prisoner’s Dilemma and the Snowdrift Game, for illustrations. Interestingly, asymmetric games reveal essential differences between models of genetic evolution based on reproduction and models of cultural evolution based on imitation that are not apparent in symmetric games.

Biological interactions, even between members of the same species, are almost always asymmetric due to differences in size, access to resources, or past interactions. However, classical game-theoretical models of evolution fail to account for sources of asymmetry in a comprehensive manner. Here, we extend the theory of evolutionary games to two general classes of asymmetry arising from environmental variation and individual differences, covering much of the heterogeneity observed in nature. If selection is weak, evolutionary processes based on asymmetric interactions behave macroscopically like symmetric games with payoffs that may depend on the resource distribution in the population or its structure. Asymmetry uncovers differences between genetic and cultural evolution that are not apparent when interactions are symmetric.

Dynamics in atomic signaling games

We study an atomic signaling game under stochastic evolutionary dynamics. There are a finite number of players who repeatedly update from a finite number of available languages/signaling strategies. Players imitate the most fit agents with high probability or mutate with low probability. We analyze the long-run distribution of states and show that, for sufficiently small mutation probability, its support is limited to efficient communication systems. We find that this behavior is insensitive to the particular choice of evolutionary dynamic, a property that is due to the game having a potential structure with a potential function corresponding to average fitness. Consequently, the model supports conclusions similar to those found in the literature on language competition. That is, we show that efficient languages eventually predominate the society while reproducing the empirical phenomenon of linguistic drift. The emergence of efficiency in the atomic case can be contrasted with results for non-atomic signaling games that establish the non-negligible possibility of convergence, under replicator dynamics, to states of unbounded efficiency loss.

Evolutionary dynamics of fairness on graphs with migration

Individual migration plays a crucial role in evolutionary dynamics of population on networks. In this paper, we generalize the networked ultimatum game by diluting population structures as well as endowing individuals with migration ability, and investigate evolutionary dynamics of fairness on graphs with migration in the ultimatum game. We first revisit the impact of node degree on the evolution of fairness. Interestingly, numerical simulations reveal that there exists an optimal value of node degree resulting in the maximal offer level of populations. Then we explore the effects of dilution and migration on the evolution of fairness, and find that both the dilution of population structures and the endowment of migration ability to individuals would lead to the drop of offer level, while the rise of acceptance level of populations. Notably, natural selection even favors the evolution of self-incompatible strategies, when either vacancy rate or migration rate exceeds a critical threshold. To confirm our simulation results, we also propose an analytical method to study the evolutionary dynamics of fairness on graphs with migration. This method can be applied to explore any games governed by pairwise interactions in finite populations.

Structure coefficients and strategy selection in multiplayer games

Evolutionary processes based on two-player games such as the Prisoner's Dilemma or Snowdrift Game are abundant in evolutionary game theory. These processes, including those based on games with more than two strategies, have been studied extensively under the assumption that selection is weak. However, games involving more than two players have not received the same level of attention. To address this issue, and to relate two-player games to multiplayer games, we introduce a notion of reducibility for multiplayer games that captures what it means to break down a multiplayer game into a sequence of interactions with fewer players. We discuss the role of reducibility in structured populations, and we give examples of games that are irreducible in any population structure. Since the known conditions for strategy selection, otherwise known as [Formula: see text]-rules, have been established only for two-player games with multiple strategies and for multiplayer games with two strategies, we extend these rules to multiplayer games with many strategies to account for irreducible games that cannot be reduced to those simpler types of games. In particular, we show that the number of structure coefficients required for a symmetric game with [Formula: see text]-player interactions and [Formula: see text] strategies grows in [Formula: see text] like [Formula: see text]. Our results also cover a type of ecologically asymmetric game based on payoff values that are derived not only from the strategies of the players, but also from their spatial positions within the population.

Aspiration dynamics in structured population acts as if in a well-mixed one

Understanding the evolution of human interactive behaviors is important. Recent experimental results suggest that human cooperation in spatial structured population is not enhanced as predicted in previous works, when payoff-dependent imitation updating rules are used. This constraint opens up an avenue to shed light on how humans update their strategies in real life. Studies via simulations show that, instead of comparison rules, self-evaluation driven updating rules may explain why spatial structure does not alter the evolutionary outcome. Though inspiring, there is a lack of theoretical result to show the existence of such evolutionary updating rule. Here we study the aspiration dynamics, and show that it does not alter the evolutionary outcome in various population structures. Under weak selection, by analytical approximation, we find that the favored strategy in regular graphs is invariant. Further, we show that this is because the criterion under which a strategy is favored is the same as that of a well-mixed population. By simulation, we show that this holds for random networks. Although how humans update their strategies is an open question to be studied, our results provide a theoretical foundation of the updating rules that may capture the real human updating rules.

Impact of heterogeneous activity and community structure on the evolutionary success of cooperators in social networks

Recent empirical studies suggest that heavy-tailed distributions of human activities are universal in real social dynamics [L. Muchnik, S. Pei, L. C. Parra, S. D. S. Reis, J. S. Andrade Jr., S. Havlin, and H. A. Makse, Sci. Rep. 3, 1783 (2013)]. On the other hand, community structure is ubiquitous in biological and social networks [M. E. J. Newman, Nat. Phys. 8, 25 (2012)]. Motivated by these facts, we here consider the evolutionary prisoner's dilemma game taking place on top of a real social network to investigate how the community structure and the heterogeneity in activity of individuals affect the evolution of cooperation. In particular, we account for a variation of the birth-death process (which can also be regarded as a proportional imitation rule from a social point of view) for the strategy updating under both weak and strong selection (meaning the payoffs harvested from games contribute either slightly or heavily to the individuals' performance). By implementing comparative studies, where the players are selected either randomly or in terms of their actual activities to play games with their immediate neighbors, we figure out that heterogeneous activity benefits the emergence of collective cooperation in a harsh environment (the action for cooperation is costly) under strong selection, whereas it impairs the formation of altruism under weak selection. Moreover, we find that the abundance of communities in the social network can evidently foster the formation of cooperation under strong selection, in contrast to the games evolving on randomized counterparts. Our results are therefore helpful for us to better understand the evolution of cooperation in real social systems.

Social imitation versus strategic choice, or consensus versus cooperation, in the networked Prisoner's Dilemma

The interplay of social and strategic motivations in human interactions is a largely unexplored topic in collective social phenomena. Whether individuals' decisions are taken in a purely strategic basis or due to social pressure without a rational background crucially influences the model outcome. Here we study a networked Prisoner's Dilemma in which decisions are made either based on the replication of the most successful neighbor's strategy (unconditional imitation) or by pure social imitation following an update rule inspired by the voter model. The main effects of the voter dynamics are an enhancement of the final consensus, i.e., asymptotic states are generally uniform, and a promotion of cooperation in certain regions of the parameter space as compared to the outcome of purely strategic updates. Thus, voter dynamics acts as an interface noise and has a similar effect as a pure random noise; furthermore, its influence is mostly independent of the network heterogeneity. When strategic decisions are made following other update rules such as the replicator or Moran processes, the dynamic mixed state found under unconditional imitation for some parameters disappears, but an increase of cooperation in certain parameter regions is still observed. Comparing our results with recent experiments on the Prisoner's Dilemma, we conclude that such a mixed dynamics may explain moody conditional cooperation among the agents.

Punishment does not promote cooperation under exploration dynamics when anti-social punishment is possible

It has been argued that punishment promotes the evolution of cooperation when mutation rates are high (i.e. when agents engage in 'exploration dynamics'). Mutations maintain a steady supply of agents that punish free-riders, and thus free-riders are at a disadvantage. Recent experiments, however, have demonstrated that free-riders sometimes also pay to punish cooperators. Inspired by these empirical results, theoretical work has explored evolutionary dynamics where mutants are rare, and found that punishment does not promote the evolution of cooperation when this 'anti-social punishment' is allowed. Here we extend previous theory by studying the effect of anti-social punishment on the evolution of cooperation across higher mutation rates, and by studying voluntary as well as compulsory Public Goods Games. We find that for intermediate and high mutation rates, adding punishment does not promote cooperation in either compulsory or voluntary public goods games if anti-social punishment is possible. This is because mutations generate agents that punish cooperators just as frequently as agents that punish defectors, and these two effects cancel each other out. These results raise questions about the effectiveness of punishment for promoting cooperation when mutations are common, and highlight how decisions about which strategies to include in the strategy set can have profound effects on the resulting dynamics.