Strategy selection in structured populations

Evolutionary dynamics of any multiplayer game on regular graphs

Multiplayer games on graphs are at the heart of theoretical descriptions of key evolutionary processes that govern vital social and natural systems. However, a comprehensive theoretical framework for solving multiplayer games with an arbitrary number of strategies on graphs is still missing. Here, we solve this by drawing an analogy with the Balls-and-Boxes problem, based on which we show that the local configuration of multiplayer games on graphs is equivalent to distributing k identical co-players among n distinct strategies. We use this to derive the replicator equation for any n-strategy multiplayer game under weak selection, which can be solved in polynomial time. As an example, we revisit the second-order free-riding problem, where costly punishment cannot truly resolve social dilemmas in a well-mixed population. Yet, in structured populations, we derive an accurate threshold for the punishment strength, beyond which punishment can either lead to the extinction of defection or transform the system into a rock-paper-scissors-like cycle. The analytical solution also qualitatively agrees with the phase diagrams that were previously obtained for non-marginal selection strengths. Our framework thus allows an exploration of any multi-strategy multiplayer game on regular graphs.

Strategy evolution on higher-order networks

Cooperation is key to prosperity in human societies. Population structure is well understood as a catalyst for cooperation, where research has focused on pairwise interactions. But cooperative behaviors are not simply dyadic, and they often involve coordinated behavior in larger groups. Here we develop a framework to study the evolution of behavioral strategies in higher-order population structures, which include pairwise and multi-way interactions. We provide an analytical treatment of when cooperation will be favored by higher-order interactions, accounting for arbitrary spatial heterogeneity and nonlinear rewards for cooperation in larger groups. Our results indicate that higher-order interactions can act to promote the evolution of cooperation across a broad range of networks, in public goods games. Higher-order interactions consistently provide an advantage for cooperation when interaction hyper-networks feature multiple conjoined communities. Our analysis provides a systematic account of how higher-order interactions modulate the evolution of prosocial traits.

Containing misinformation: Modeling spatial games of fake news

The spread of fake news on social media is a pressing issue. Here, we develop a mathematical model on social networks in which news sharing is modeled as a coordination game. We use this model to study the effect of adding designated individuals who sanction fake news sharers (representing, for example, correction of false claims or public shaming of those who share such claims). By simulating our model on synthetic square lattices and small-world networks, we demonstrate that social network structure allows fake news spreaders to form echo chambers and more than doubles fake news' resistance to distributed sanctioning efforts. We confirm our results are robust to a wide range of coordination and sanctioning payoff parameters as well as initial conditions. Using a Twitter network dataset, we show that sanctioners can help contain fake news when placed strategically. Furthermore, we analytically determine the conditions required for peer sanctioning to be effective, including prevalence and enforcement levels. Our findings have implications for developing mitigation strategies to control misinformation and preserve the integrity of public discourse.

Linked selection and the evolution of altruism in family-structured populations

Much research on the evolution of altruism via kin selection, group selection, and reciprocity focuses on the role of a single locus or quantitative trait. Very few studies have explored how linked selection, or selection at loci neighboring an altruism locus, impacts the evolution of altruism. While linked selection can decrease the efficacy of selection at neighboring loci, it might have other effects including promoting selection for altruism by increasing relatedness in regions of low recombination. Here, we used population genetic simulations to study how negative selection at linked loci, or background selection, affects the evolution of altruism. When altruism occurs between full siblings, we found that background selection interfered with selection on the altruistic allele, increasing its fixation probability when the altruistic allele was disfavored and reducing its fixation when the allele was favored. In other words, background selection has the same effect on altruistic genes in family-structured populations as it does on other, nonsocial, genes. This contrasts with prior research showing that linked selective sweeps can favor the evolution of cooperation, and we discuss possibilities for resolving these contrasting results.

Imitation dynamics on networks with incomplete information

Imitation is an important learning heuristic in animal and human societies. Previous explorations report that the fate of individuals with cooperative strategies is sensitive to the protocol of imitation, leading to a conundrum about how different styles of imitation quantitatively impact the evolution of cooperation. Here, we take a different perspective on the personal and external social information required by imitation. We develop a general model of imitation dynamics with incomplete information in networked systems, which unifies classical update rules including the death-birth and pairwise-comparison rule on complex networks. Under pairwise interactions, we find that collective cooperation is most promoted if individuals neglect personal information. If personal information is considered, cooperators evolve more readily with more external information. Intriguingly, when interactions take place in groups on networks with low degrees of clustering, using more personal and less external information better facilitates cooperation. Our unifying perspective uncovers intuition by examining the rate and range of competition induced by different information situations.

Nudging cooperation among agents in an experimental social network

We investigate the development of cooperative behavior in networks over time. In our controlled laboratory experiment, subjects can cooperate by sending costly messages that contain valuable information for the receiver or other subjects in the network. Any message sent can increase the chance that subjects find the information they are looking for and consequently their profit. We find that cooperation emerges spontaneously and remains stable over time. In an additional treatment, we provide a non-binding suggestion about who to contact at the beginning of the experiment. We find that subjects partially follow our recommendation, and this increases their own and others' profit. Despite the removal of suggestions, subjects build long-lasting relationships with the suggested contacts.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41109-023-00588-x.

Strategy evolution on dynamic networks

Models of strategy evolution on static networks help us understand how population structure can promote the spread of traits like cooperation. One key mechanism is the formation of altruistic spatial clusters, where neighbors of a cooperative individual are likely to reciprocate, which protects prosocial traits from exploitation. However, most real-world interactions are ephemeral and subject to exogenous restructuring, so that social networks change over time. Strategic behavior on dynamic networks is difficult to study, and much less is known about the resulting evolutionary dynamics. Here we provide an analytical treatment of cooperation on dynamic networks, allowing for arbitrary spatial and temporal heterogeneity. We show that transitions among a large class of network structures can favor the spread of cooperation, even if each individual social network would inhibit cooperation when static. Furthermore, we show that spatial heterogeneity tends to inhibit cooperation, whereas temporal heterogeneity tends to promote it. Dynamic networks can have profound effects on the evolution of prosocial traits, even when individuals have no agency over network structures.

Evolutionary dynamics on sequential temporal networks

Population structure is a well-known catalyst for the evolution of cooperation and has traditionally been considered to be static in the course of evolution. Conversely, real-world populations, such as microbiome communities and online social networks, frequently show a progression from tiny, active groups to huge, stable communities, which is insufficient to be captured by constant structures. Here, we propose sequential temporal networks to characterize growing networked populations, and we extend the theory of evolutionary games to these temporal networks with arbitrary structures and growth rules. We derive analytical rules under which a sequential temporal network has a higher fixation probability for cooperation than its static counterpart. Under neutral drift, the rule is simply a function of the increment of nodes and edges in each time step. But if the selection is weak, the rule is related to coalescence times on networks. In this case, we propose a mean-field approximation to calculate fixation probabilities and critical benefit-to-cost ratios with lower calculation complexity. Numerical simulations in empirical datasets also prove the cooperation-promoting effect of population growth. Our research stresses the significance of population growth in the real world and provides a high-accuracy approximation approach for analyzing the evolution in real-life systems.

Heterogeneous contributions can jeopardize cooperation in the public goods game

When studying social dilemma games, a crucial question arises regarding the impact of general heterogeneity on cooperation, which has been shown to have positive effects in numerous studies. Here, we demonstrate that heterogeneity in the contribution value for the focal public goods game can jeopardize cooperation. We show that there is an optimal contribution value in the homogeneous case that most benefits cooperation depending on the lattice. In a heterogeneous scenario, where strategy and contribution coevolve, cooperators making contributions higher than the optimal value end up harming those who contribute less. This effect is notably detrimental to cooperation in the square lattice with von Neumann neighborhood, while it can have no impact in other lattices. Furthermore, in parameter regions where a higher-contributing cooperator cannot normally survive alone, the exploitation of lower-value contribution cooperators allows their survival, resembling a parasitic behavior. To obtain these results, we examined the effect of various distributions for the contribution values in the initial condition and we conducted Monte Carlo simulations.

Fifty shades of greenbeard: robust evolution of altruism based on similarity of complex phenotypes

We study the evolution of altruistic behaviour under a model where individuals choose to cooperate by comparing a set of continuous phenotype tags. Individuals play a donation game and only donate to other individuals that are sufficiently similar to themselves in a multidimensional phenotype space. We find the generic maintenance of robust altruism when phenotypes are multidimensional. Selection for altruism is driven by the coevolution of individual strategy and phenotype; altruism levels shape the distribution of individuals in phenotype space. Low donation rates induce a phenotype distribution that renders the population vulnerable to the invasion of altruists, whereas high donation rates prime a population for cheater invasion, resulting in cyclic dynamics that maintain substantial levels of altruism. Altruism is therefore robust to invasion by cheaters in the long term in this model. Furthermore, the shape of the phenotype distribution in high phenotypic dimension allows altruists to better resist the invasion by cheaters, and as a result the amount of donation increases with increasing phenotype dimension. We also generalize previous results in the regime of weak selection to two competing strategies in continuous phenotype space, and show that success under weak selection is crucial to success under strong selection in our model. Our results support the viability of a simple similarity-based mechanism for altruism in a well-mixed population.

The future of theoretical evolutionary game theory

Evolutionary game theory is a truly interdisciplinary subject that goes well beyond the limits of biology. Mathematical minds get hooked up in simple models for evolution and often gradually move into other parts of evolutionary biology or ecology. Social scientists realize how much they can learn from evolutionary thinking and gradually transfer insight that was originally generated in biology. Computer scientists can use their algorithms to explore a new field where machines not only learn from the environment, but also from each other. The breadth of the field and the focus on a few very popular issues, such as cooperation, comes at a price: several insights are re-discovered in different fields under different labels with different heroes and modelling traditions. For example, reciprocity or spatial structure are treated differently. Will we continue to develop things in parallel? Or can we converge to a single set of ideas, a single tradition and eventually a single software repository? Or will these fields continue to cross-fertilize each other, learning from each other and engaging in a constructive exchange between fields? Ultimately, the popularity of evolutionary game theory rests not only on its explanatory power, but also on the intuitive character of its models. This article is part of the theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions'.

Reproductive variance can drive behavioral dynamics

The concept of fitness is central to evolution, but it quantifies only the expected number of offspring an individual will produce. The actual number of offspring is also subject to demographic stochasticity-that is, randomness associated with birth and death processes. In nature, individuals who are more fecund tend to have greater variance in their offspring number. Here, we develop a model for the evolution of two types competing in a population of nonconstant size. The fitness of each type is determined by pairwise interactions in a prisoner's dilemma game, and the variance in offspring number depends upon its mean. Although defectors are preferred by natural selection in classical population models, since they always have greater fitness than cooperators, we show that sufficiently large offspring variance can reverse the direction of evolution and favor cooperation. Large offspring variance produces qualitatively new dynamics for other types of social interactions, as well, which cannot arise in populations with a fixed size or with a Poisson offspring distribution.

Optimization of institutional incentives for cooperation in structured populations

The application of incentives, such as reward and punishment, is a frequently applied way for promoting cooperation among interacting individuals in structured populations. However, how to properly use the incentives is still a challenging problem for incentive-providing institutions. In particular, since the implementation of incentive is costly, to explore the optimal incentive protocol, which ensures the desired collective goal at a minimal cost, is worthy of study. In this work, we consider the positive and negative incentives for a structured population of individuals whose conflicting interactions are characterized by a Prisoner's Dilemma game. We establish an index function for quantifying the cumulative cost during the process of incentive implementation, and theoretically derive the optimal positive and negative incentive protocols for cooperation on regular networks. We find that both types of optimal incentive protocols are identical and time-invariant. Moreover, we compare the optimal rewarding and punishing schemes concerning implementation cost and provide a rigorous basis for the usage of incentives in the game-theoretical framework. We further perform computer simulations to support our theoretical results and explore their robustness for different types of population structures, including regular, random, small-world and scale-free networks.

Evolution of cooperation under a generalized death-birth process

According to the evolutionary death-birth protocol, a player is chosen randomly to die and neighbors compete for the available position proportional to their fitness. Hence, the status of the focal player is completely ignored and has no impact on the strategy update. In this paper, we revisit and generalize this rule by introducing a weight factor to compare the payoff values of the focal and invading neighbors. By means of evolutionary graph theory, we analyze the model on joint transitive graphs to explore the possible consequences of the presence of a weight factor. We find that focal weight always hinders cooperation under weak selection strength. Surprisingly, the results show a nontrivial tipping point of the weight factor where the threshold of cooperation success shifts from positive to negative infinity. Once focal weight exceeds this tipping point, cooperation becomes unreachable. Our theoretical predictions are confirmed by Monte Carlo simulations on a square lattice of different sizes. We also verify the robustness of the conclusions to arbitrary two-player prisoner's dilemmas, to dispersal graphs with arbitrary edge weights, and to interaction and dispersal graphs overlapping arbitrarily.

Evolutionary dynamics under partner preferences

The fact that people often have preference rankings for their partners is a distinctive aspect of human behavior. Little is known, however, about how this talent as a powerful force shapes human behavioral traits, including those which should not have been favored by selection, such as cooperation in social dilemma situations. Here we propose a dynamic model in which network-structured individuals can switch their interaction partners within neighborhoods based on their preferences. For the partner switching, we propose two interruption regimes: dictatorial regime and negotiating regime. In the dictatorial regime, focal individuals are able to suspend interactions out of preferences unilaterally. In the negotiating regime, either focal individuals or the associated partners agree to suspend, then these interactions can be successfully suspended. We investigate the evolution of cooperation under both preference-driven partner switching regimes in the context of both the weakened variant of the donation game and the standard one. Specifically, we theoretically approximate the critical conditions for cooperation to be favored by weak selection in the weakened donation game where cooperators bear a unit cost to provide a benefit for each active neighbor and simulate the evolutionary dynamics of cooperation in the standard donation game to test the robustness of the analytical results. Under dictatorial regime, selection of cooperation becomes harder when individuals have preferences for either cooperator or defector partners, implying that the expulsion of defectors by cooperators is overwhelmed by the chasing of defectors towards cooperators. Under negotiating regime, both preferences for cooperator and defector partners can significantly favor the evolution of cooperation, yet underlying mechanisms differ greatly. For preferences over cooperator partners, cooperator-cooperator interaction relationships are reinforced and the associated mutual reciprocity can resist and assimilate defectors. For preferences over defector partners, defector-defector interaction relationships are anchored, weakening defectors' exploitation over cooperators. Cooperators are thus offered much time space to interact among cospecies and spread. Our work may help better understand the critical role of preference-based adaptive partner switching in promoting the evolution of cooperation.

Altruistic behaviors and cooperation among gifted adolescents

The present study is a differential study that describes the nature of the relationship between cooperation and altruistic behavior in a sample of gifted adolescents in three universities in Egypt and Kuwait University. It also identified the differences between males/females, and senior students/junior students in both cooperation and altruism. A total of 237 gifted adolescents—with average age 21.3 ± SD 2.6 years—from three Egyptian universities: Alexandria University, Sadat Academy for Management Sciences, and Suez University (in Egypt), and Kuwait University, were involved in this study. Measures used in the study include the Scales for Rating the Behavioral Characteristics of Superior Students (SRBCSS), Generative Altruism Scale (GAlS), and The Cooperative/Competitive Strategy Scale (CCSS). Results revealed that there is a significant positive relationship between altruism and cooperation among gifted adolescents. Also, findings show that there are statistically significant differences between males and females in both altruism and cooperation. In addition, there are differences statistically significant between senior students and junior students in both altruism and cooperation in favor of senior students. It is recommended that altruism and cooperation intervention-based programs should be designed to increase the adaptive behaviors of adolescents.

The role of evolutionary game theory in spatial and non-spatial models of the survival of cooperation in cancer: a review

Evolutionary game theory (EGT) is a branch of mathematics which considers populations of individuals interacting with each other to receive pay-offs. An individual’s pay-off is dependent on the strategy of its opponent(s) as well as on its own, and the higher its pay-off, the higher its reproductive fitness. Its offspring generally inherit its interaction strategy, subject to random mutation. Over time, the composition of the population shifts as different strategies spread or are driven extinct. In the last 25 years there has been a flood of interest in applying EGT to cancer modelling, with the aim of explaining how cancerous mutations spread through healthy tissue and how intercellular cooperation persists in tumour-cell populations. This review traces this body of work from theoretical analyses of well-mixed infinite populations through to more realistic spatial models of the development of cooperation between epithelial cells. We also consider work in which EGT has been used to make experimental predictions about the evolution of cancer, and discuss work that remains to be done before EGT can make large-scale contributions to clinical treatment and patient outcomes.

Evaluating the structure-coefficient theorem of evolutionary game theory

In order to accommodate the empirical fact that population structures are rarely simple, modern studies of evolutionary dynamics allow for complicated and highly heterogeneous spatial structures. As a result, one of the most difficult obstacles lies in making analytical deductions, either qualitative or quantitative, about the long-term outcomes of evolution. The "structure-coefficient" theorem is a well-known approach to this problem for mutation-selection processes under weak selection, but a general method of evaluating the terms it comprises is lacking. Here, we provide such a method for populations of fixed (but arbitrary) size and structure, using easily interpretable demographic measures. This method encompasses a large family of evolutionary update mechanisms and extends the theorem to allow for asymmetric contests to provide a better understanding of the mutation-selection balance under more realistic circumstances. We apply the method to study social goods produced and distributed among individuals in spatially heterogeneous populations, where asymmetric interactions emerge naturally and the outcome of selection varies dramatically, depending on the nature of the social good, the spatial topology, and the frequency with which mutations arise.

When stochasticity leads to cooperation

The evolution of cooperation has gained more attention after Smith introduced game theory in the study of evolutionary biology. Subsequent works have extensively explained this phenomenon, consistently showing the importance of spatial structure for the evolution of cooperation. Here we analyze the effect of stochasticity on the evolution of cooperation in group-structured populations. We find a simple formula for the fixation probability of cooperators and show that cooperation can be favored by selection if a condition similar to Hamilton's rule is satisfied, which is also valid for strong selection and high migration. In fact, cooperation can be favored even in the absence of population viscosity and in the limit of an infinite number of finite-size groups. We discuss the importance of stochastic fluctuations in helping cooperation. We argue that this may be a general principle because fluctuations favoring the cooperators are often much more impactful than those favoring the defectors.

Evolution of cooperation with contextualized behavior

How do networks of social interaction govern the emergence and stability of prosocial behavior? Theoretical studies of this question typically assume unconditional behavior, meaning that an individual either cooperates with all opponents or defects against all opponents-an assumption that produces a pessimistic outlook for the evolution of cooperation, especially in highly connected populations. Although these models may be appropriate for simple organisms, humans have sophisticated cognitive abilities that allow them to distinguish between opponents and social contexts, so they can condition their behavior on the identity of opponents. Here, we study the evolution of cooperation when behavior is conditioned by social context, but behaviors can spill over between contexts. Our mathematical analysis shows that contextualized behavior rescues cooperation across a broad range of population structures, even when the number of social contexts is small. Increasing the number of social contexts further promotes cooperation by orders of magnitude.

Evolution of cooperation with asymmetric social interactions

How cooperation emerges in human societies is both an evolutionary enigma and a practical problem with tangible implications for societal health. Population structure has long been recognized as a catalyst for cooperation because local interactions facilitate reciprocity. Analysis of population structure typically assumes bidirectional social interactions. But human social interactions are often unidirectional-where one individual has the opportunity to contribute altruistically to another, but not conversely-as the result of organizational hierarchies, social stratification, popularity effects, and endogenous mechanisms of network growth. Here we expand the theory of cooperation in structured populations to account for both uni- and bidirectional social interactions. Even though unidirectional interactions remove the opportunity for reciprocity, we find that cooperation can nonetheless be favored in directed social networks and that cooperation is provably maximized for networks with an intermediate proportion of unidirectional interactions, as observed in many empirical settings. We also identify two simple structural motifs that allow efficient modification of interaction directions to promote cooperation by orders of magnitude. We discuss how our results relate to the concepts of generalized and indirect reciprocity.

Evolution of prosocial behaviours in multilayer populations

Human societies include diverse social relationships. Friends, family, business colleagues and online contacts can all contribute to one's social life. Individuals may behave differently in different domains, but success in one domain may engender success in another. Here, we study this problem using multilayer networks to model multiple domains of social interactions, in which individuals experience different environments and may express different behaviours. We provide a mathematical analysis and find that coupling between layers tends to promote prosocial behaviour. Even if prosociality is disfavoured in each layer alone, multilayer coupling can promote its proliferation in all layers simultaneously. We apply this analysis to six real-world multilayer networks, ranging from the socio-emotional and professional relationships in a Zambian community, to the online and offline relationships within an academic university. We discuss the implications of our results, which suggest that small modifications to interactions in one domain may catalyse prosociality in a different domain.

Interindividual cooperation mediated by partisanship complicates Madison's cure for "mischiefs of faction"

Political theorists have long argued that enlarging the political sphere to include a greater diversity of interests would cure the ills of factions in a pluralistic society. While the scope of politics has expanded dramatically over the past 75 y, polarization is markedly worse. Motivated by this paradox, we take a bottom-up approach to explore how partisan individual-level dynamics in a diverse (multidimensional) issue space can shape collective-level factionalization via an emergent dimensionality reduction. We extend a model of cultural evolution grounded in evolutionary game theory, in which individuals accumulate benefits through pairwise interactions and imitate (or learn) the strategies of successful others. The degree of partisanship determines the likelihood of learning from individuals of the opposite party. This approach captures the coupling between individual behavior, partisan-mediated opinion dynamics, and an interaction network that changes endogenously according to the evolving interests of individuals. We find that while expanding the diversity of interests can indeed improve both individual and collective outcomes, increasingly high partisan bias promotes a reduction in issue dimensionality via party-based assortment that leads to increasing polarization. When party bias becomes extreme, it also boosts interindividual cooperation, thereby further entrenching extreme polarization and creating a tug-of-war between individual cooperation and societal cohesion. These dangers of extreme partisanship are highest when individuals' interests and opinions are heavily shaped by peers and there is little independent exploration. Overall, our findings highlight the urgency to study polarization in a coupled, multilevel context.

Modeling pluralism and self-regulation explains the emergence of cooperation in networked societies

Understanding the dynamics of cooperative behavior of individuals in complex societies represents a fundamental research question which puzzles scientists working in heterogeneous fields. Many studies have been developed using the unitary agent assumption, which embeds the idea that when making decisions, individuals share the same socio-cultural parameters. In this paper, we propose the ECHO-EGN model, based on Evolutionary Game Theory, which relaxes this strong assumption by considering the heterogeneity of three fundamental socio-cultural aspects ruling the behavior of groups of people: the propensity to be more cooperative with members of the same group (Endogamic cooperation), the propensity to cooperate with the public domain (Civicness) and the propensity to prefer connections with members of the same group (Homophily). The ECHO-EGN model is shown to have high performance in describing real world behavior of interacting individuals living in complex environments. Extensive numerical experiments allowing the comparison of real data and model simulations confirmed that the introduction of the above mechanisms enhances the realism in the modelling of cooperation dynamics. Additionally, theoretical findings allow us to conclude that endogamic cooperation may limit significantly the emergence of cooperation.

Cooperative success in epithelial public goods games

Cancer cells obtain mutations which rely on the production of diffusible growth factors to confer a fitness benefit. These mutations can be considered cooperative, and studied as public goods games within the framework of evolutionary game theory. The population structure, benefit function and update rule all influence the evolutionary success of cooperators. We model the evolution of cooperation in epithelial cells using the Voronoi tessellation model. Unlike traditional evolutionary graph theory, this allows us to implement global updating, for which birth and death events are spatially decoupled. We compare, for a sigmoid benefit function, the conditions for cooperation to be favoured and/or beneficial for well-mixed and structured populations. We find that when population structure is combined with global updating, cooperation is more successful than if there were local updating or the population were well-mixed. Interestingly, the qualitative behaviour for the well-mixed population and the Voronoi tessellation model is remarkably similar, but the latter case requires significantly lower incentives to ensure cooperation.

Evolution of state-dependent strategies in stochastic games

In a population of interacting individuals, the environment for interactions often changes due to individuals' behaviors, which in turn drive the evolution of individuals' behaviors. The interplay between the environment and individuals' behaviors has been demonstrated to remarkably influence the evolutionary outcomes. In reality, in highly cognitive species such as social primates and human beings, individuals are often capable of perceiving the environment change and then differentiate their strategies across different environment states. We propose a model of environmental feedback with state-dependent strategies: individuals have perceptions of distinct environment states and therefore take distinct sub-strategies under each of them; based on the sub-strategy, individuals then decide their behaviors; their behaviors subsequently modify the environment state. We use the theory of stochastic games and evolutionary dynamics to analyze this idea. We find that when environment changes slower than behaviors, state-dependent strategies (i.e. taking different sub-strategies under different environment states) can outperform state-independent strategies (i.e. taking an identical sub-strategy under all environment states), such as Win-Stay, Lose-Shift, the most leading strategy among state-independent strategies. The intuition is that delayed environmental feedback provides chances for individuals with state-dependent strategies to exploit those with state-independent strategies. Our results hold (1) in both well-mixed and structured populations; (2) when the environment switches between two or more states. Furthermore, the environment changing rate decides if state-dependent strategies benefit global cooperation. The evolution sees the rise of the cooperation level for fast environment switching and the decrease otherwise. Our work stresses that individuals' perceptions of different environment states are beneficial to their survival and social prosperity in a changing world.

Aspiration dynamics generate robust predictions in heterogeneous populations

Update rules, which describe how individuals adjust their behavior over time, affect the outcome of social interactions. Theoretical studies have shown that evolutionary outcomes are sensitive to model details when update rules are imitation-based but are robust when update rules are self-evaluation based. However, studies of self-evaluation based rules have focused on homogeneous population structures where each individual has the same number of neighbors. Here, we consider heterogeneous population structures represented by weighted networks. Under weak selection, we analytically derive the condition for strategy success, which coincides with the classical condition of risk-dominance. This condition holds for all weighted networks and distributions of aspiration levels, and for individualized ways of self-evaluation. Our findings recover previous results as special cases and demonstrate the universality of the robustness property under self-evaluation based rules. Our work thus sheds light on the intrinsic difference between evolutionary dynamics under self-evaluation based and imitation-based update rules.

Social interaction outcomes can depend on the type of information individuals possess and how it is used in decision-making. Here, Zhou et al. find that self-evaluation based decision-making rules lead to evolutionary outcomes that are robust to different population structures and ways of self-evaluation.

A game-theoretical dynamic imitation model on networks

A game-theoretical model is constructed to capture the effect of imitation on the evolution of cooperation. This imitation describes the case where successful individuals are more likely to be imitated by newcomers who will employ their strategies and social networks. Two classical repeated strategies 'always defect (ALLD)' and 'tit-for-tat (TFT)' are adopted. Mathematical analyses are mainly conducted by the method of coalescence theory. Under the assumption of a large population size and weak selection, the results show that the evolution of cooperation is promoted in this dynamic network. As we observed that the critical benefit-to-cost ratio is smaller compared to that in well-mixed populations. The critical benefit-to-cost ratio approaches a specific value which depends on three parameters, the repeated rounds of the game, the effective strategy mutation rate, and the effective link mutation rate. Specifically, for a very high value of the effective link mutation rate, the critical benefit-to-cost ratio approaches 1. Remarkably, for a low value of the effective link mutation rate, by letting the effective strategy mutation is nearly equal to zero, the critical benefit-to-cost ratio approaches [Formula: see text] for the resulting highly connected networks, which allows TFT to be evolutionary stable. It illustrates that dominance of TFTs is associated with more connected networks. This research can enrich the theory of the coevolution of game strategy and network structure with dynamic imitation.

Fixation probabilities in evolutionary dynamics under weak selection

In evolutionary dynamics, a key measure of a mutant trait's success is the probability that it takes over the population given some initial mutant-appearance distribution. This "fixation probability" is difficult to compute in general, as it depends on the mutation's effect on the organism as well as the population's spatial structure, mating patterns, and other factors. In this study, we consider weak selection, which means that the mutation's effect on the organism is small. We obtain a weak-selection perturbation expansion of a mutant's fixation probability, from an arbitrary initial configuration of mutant and resident types. Our results apply to a broad class of stochastic evolutionary models, in which the size and spatial structure are arbitrary (but fixed). The problem of whether selection favors a given trait is thereby reduced from exponential to polynomial complexity in the population size, when selection is weak. We conclude by applying these methods to obtain new results for evolutionary dynamics on graphs.

Learning enables adaptation in cooperation for multi-player stochastic games

Interactions among individuals in natural populations often occur in a dynamically changing environment. Understanding the role of environmental variation in population dynamics has long been a central topic in theoretical ecology and population biology. However, the key question of how individuals, in the middle of challenging social dilemmas (e.g. the 'tragedy of the commons'), modulate their behaviours to adapt to the fluctuation of the environment has not yet been addressed satisfactorily. Using evolutionary game theory, we develop a framework of stochastic games that incorporates the adaptive mechanism of reinforcement learning to investigate whether cooperative behaviours can evolve in the ever-changing group interaction environment. When the action choices of players are just slightly influenced by past reinforcements, we construct an analytical condition to determine whether cooperation can be favoured over defection. Intuitively, this condition reveals why and how the environment can mediate cooperative dilemmas. Under our model architecture, we also compare this learning mechanism with two non-learning decision rules, and we find that learning significantly improves the propensity for cooperation in weak social dilemmas, and, in sharp contrast, hinders cooperation in strong social dilemmas. Our results suggest that in complex social-ecological dilemmas, learning enables the adaptation of individuals to varying environments.

Evolution of cooperation in an epithelium

Cooperation is prevalent in nature, not only in the context of social interactions within the animal kingdom but also on the cellular level. In cancer, for example, tumour cells can cooperate by producing growth factors. The evolution of cooperation has traditionally been studied for well-mixed populations under the framework of evolutionary game theory, and more recently for structured populations using evolutionary graph theory (EGT). The population structures arising due to cellular arrangement in tissues, however, are dynamic and thus cannot be accurately represented by either of these frameworks. In this work, we compare the conditions for cooperative success in an epithelium modelled using EGT, to those in a mechanical model of an epithelium—the Voronoi tessellation (VT) model. Crucially, in this latter model, cells are able to move, and birth and death are not spatially coupled. We calculate fixation probabilities in the VT model through simulation and an approximate analytic technique and show that this leads to stronger promotion of cooperation in comparison with the EGT model.

Spatial reciprocity in the evolution of cooperation

Cooperation is key to the survival of all biological systems. The spatial structure of a system constrains who interacts with whom (interaction partner) and who acquires new traits from whom (role model). Understanding when and to what degree a spatial structure affects the evolution of cooperation is an important and challenging topic. Here, we provide an analytical formula to predict when natural selection favours cooperation where the effects of a spatial structure are described by a single parameter. We find that a spatial structure promotes cooperation (spatial reciprocity) when interaction partners overlap role models. When they do not, spatial structure inhibits cooperation even without cooperation dilemmas. Furthermore, a spatial structure in which individuals interact with their role models more often shows stronger reciprocity. Thus, imitating individuals with frequent interactions facilitates cooperation. Our findings are applicable to both pairwise and group interactions and show that strong social ties might hinder, while asymmetric spatial structures for interaction and trait dispersal could promote cooperation.

Deconstructing Evolutionary Game Theory: Coevolution of Social Behaviors with Their Evolutionary Setting

Evolution of social behaviors is one of the most fascinating and active fields of evolutionary biology. During the past half century, social evolution theory developed into a mature field with powerful tools to understand the dynamics of social traits such as cooperation under a wide range of conditions. In this article, I argue that the next stage in the development of social evolution theory should consider the evolution of the setting in which social behaviors evolve. To that end, I propose a conceptual map of the components that make up the evolutionary setting of social behaviors, review existing work that considers the evolution of each component, and discuss potential future directions. The theoretical work reviewed here illustrates how unexpected dynamics can happen when the setting of social evolution itself is evolving, such as cooperation sometimes being self-limiting. I argue that a theory of how the setting of social evolution itself evolves will lead to a deeper understanding of when cooperation and other social behaviors evolve and diversify.

Evolutionary dynamics with game transitions

The environment has a strong influence on a population's evolutionary dynamics. Driven by both intrinsic and external factors, the environment is subject to continual change in nature. To capture an ever-changing environment, we consider a model of evolutionary dynamics with game transitions, where individuals' behaviors together with the games that they play in one time step influence the games to be played in the next time step. Within this model, we study the evolution of cooperation in structured populations and find a simple rule: Weak selection favors cooperation over defection if the ratio of the benefit provided by an altruistic behavior, b, to the corresponding cost, c, exceeds [Formula: see text], where k is the average number of neighbors of an individual and [Formula: see text] captures the effects of the game transitions. Even if cooperation cannot be favored in each individual game, allowing for a transition to a relatively valuable game after mutual cooperation and to a less valuable game after defection can result in a favorable outcome for cooperation. In particular, small variations in different games being played can promote cooperation markedly. Our results suggest that simple game transitions can serve as a mechanism for supporting prosocial behaviors in highly connected populations.

Evolutionary games on isothermal graphs

Population structure affects the outcome of natural selection. These effects can be modeled using evolutionary games on graphs. Recently, conditions were derived for a trait to be favored under weak selection, on any weighted graph, in terms of coalescence times of random walks. Here we consider isothermal graphs, which have the same total edge weight at each node. The conditions for success on isothermal graphs take a simple form, in which the effects of graph structure are captured in the ‘effective degree’—a measure of the effective number of neighbors per individual. For two update rules (death-Birth and birth-Death), cooperative behavior is favored on a large isothermal graph if the benefit-to-cost ratio exceeds the effective degree. For two other update rules (Birth-death and Death-birth), cooperation is never favored. We relate the effective degree of a graph to its spectral gap, thereby linking evolutionary dynamics to the theory of expander graphs. Surprisingly, we find graphs of infinite average degree that nonetheless provide strong support for cooperation.

The spatial structure of a population is often critical for the evolution of cooperation. Here, Allen and colleagues show that when spatial structure is represented by an isothermal graph, the effective number of neighbors per individual determines whether or not cooperation can evolve.

Cooperation in Microbial Populations: Theory and Experimental Model Systems

Cooperative behavior, the costly provision of benefits to others, is common across all domains of life. This review article discusses cooperative behavior in the microbial world, mediated by the exchange of extracellular products called public goods. We focus on model species for which the production of a public good and the related growth disadvantage for the producing cells are well described. To unveil the biological and ecological factors promoting the emergence and stability of cooperative traits we take an interdisciplinary perspective and review insights gained from both mathematical models and well-controlled experimental model systems. Ecologically, we include crucial aspects of the microbial life cycle into our analysis and particularly consider population structures where ensembles of local communities (subpopulations) continuously emerge, grow, and disappear again. Biologically, we explicitly consider the synthesis and regulation of public good production. The discussion of the theoretical approaches includes general evolutionary concepts, population dynamics, and evolutionary game theory. As a specific but generic biological example, we consider populations of Pseudomonas putida and its regulation and use of pyoverdines, iron scavenging molecules, as public goods. The review closes with an overview on cooperation in spatially extended systems and also provides a critical assessment of the insights gained from the experimental and theoretical studies discussed. Current challenges and important new research opportunities are discussed, including the biochemical regulation of public goods, more realistic ecological scenarios resembling native environments, cell-to-cell signaling, and multispecies communities.

Computation and Simulation of Evolutionary Game Dynamics in Finite Populations

The study of evolutionary dynamics increasingly relies on computational methods, as more and more cases outside the range of analytical tractability are explored. The computational methods for simulation and numerical approximation of the relevant quantities are diverging without being compared for accuracy and performance. We thoroughly investigate these algorithms in order to propose a reliable standard. For expositional clarity we focus on symmetric 2 × 2 games leading to one-dimensional processes, noting that extensions can be straightforward and lessons will often carry over to more complex cases. We provide time-complexity analysis and systematically compare three families of methods to compute fixation probabilities, fixation times and long-term stationary distributions for the popular Moran process. We provide efficient implementations that substantially improve wall times over naive or immediate implementations. Implications are also discussed for the Wright-Fisher process, as well as structured populations and multiple types.

Evolutionary multiplayer games on graphs with edge diversity

Evolutionary game dynamics in structured populations has been extensively explored in past decades. However, most previous studies assume that payoffs of individuals are fully determined by the strategic behaviors of interacting parties, and social ties between them only serve as the indicator of the existence of interactions. This assumption neglects important information carried by inter-personal social ties such as genetic similarity, geographic proximity, and social closeness, which may crucially affect the outcome of interactions. To model these situations, we present a framework of evolutionary multiplayer games on graphs with edge diversity, where different types of edges describe diverse social ties. Strategic behaviors together with social ties determine the resulting payoffs of interactants. Under weak selection, we provide a general formula to predict the success of one behavior over the other. We apply this formula to various examples which cannot be dealt with using previous models, including the division of labor and relationship- or edge-dependent games. We find that labor division can promote collective cooperation markedly. The evolutionary process based on relationship-dependent games can be approximated by interactions under a transformed and unified game. Our work stresses the importance of social ties and provides effective methods to reduce the calculating complexity in analyzing the evolution of realistic systems.

The outcome of an interaction often relies on not only interactants’ strategic behaviors but also genetic and physical relationships between interactants, such as genetic similarity and geographic proximity. Thus when encountering different opponents who use the same strategy, an individual may derive different payoffs. Social ties, acting as carriers of such information, are crucial to biological interactions. However, most prior studies simplify social ties as binary states (i.e., either present or absent) and ignore the information carried. Here we study evolutionary multiplayer games on graphs and introduce different types of edges to describe diverse social ties. We derive a simple rule to predict when a strategic behavior is more successful than the other. Based on this rule, we find that the labor division in eusocial insects could promote prosocial behavior. In addition, when payoff structures in different interactions are relationship-dependent, the condition for the success of one behavior can be obtained by studying interactions described by a unified payoff structure. Our work not only extends established results on the evolution of cooperation on graphs, but also shows the possibility to simplify complex and diverse interactions in real-world systems as simple and unified interactions in theoretical calculations.

Evolution of cooperation on large networks with community structure

Cooperation is a major factor in the evolution of human societies. The structure of social networks, which affects the dynamics of cooperation and other interpersonal phenomena, have common structural signatures. One of these signatures is the tendency to organize as groups. This tendency gives rise to networks with community structure, which are composed of distinct modules. In this paper, we study analytically the evolutionary game dynamics on large modular networks in the limit of weak selection. We obtain novel analytical conditions such that natural selection favours cooperation over defection. We calculate the transition point for each community to favour cooperation. We find that a critical inter-community link creation probability exists for given group density, such that the overall network supports cooperation even if individual communities inhibit it. As a byproduct, we present solutions for the critical benefit-to-cost ratio which perform with remarkable accuracy for diverse generative network models, including those with community structure and heavy-tailed degree distributions. We also demonstrate the generalizability of the results to arbitrary two-player games.

Properties of network structures, structure coefficients, and benefit-to-cost ratios

In structured populations the spatial arrangement of cooperators and defectors on the interaction graph together with the structure of the graph itself determines the game dynamics and particularly whether or not fixation of cooperation (or defection) is favored. For networks described by regular graphs and for a single cooperator (and a single defector) the question of fixation can be addressed by a single parameter, the structure coefficient. This quantity is invariant with respect to the location of the cooperator on the graph and also does not vary over different networks. We may therefore consider it to be generic for regular graphs and call it the generic structure coefficient. For two and more cooperators (or several defectors) fixation properties can also be assigned by structure coefficients. These structure coefficients, however, depend on the arrangement of cooperators and defectors which we may interpret as a configuration of the game. Moreover, the coefficients are specific for a given interaction network modeled as a regular graph, which is why we may call them specific structure coefficients. In this paper, we study how specific structure coefficients vary over interaction graphs and analyze how spectral properties of interaction networks relate to specific structure coefficients. We also discuss implications for the benefit-to-cost ratios of donation games.

Fixation properties of multiple cooperator configurations on regular graphs

Whether or not cooperation is favored in evolutionary games on graphs depends on the population structure and spatial properties of the interaction network. The population structure can be expressed as configurations. Such configurations extend scenarios with a single cooperator among defectors to any number of cooperators and any arrangement of cooperators and defectors on the network. For interaction networks modeled as regular graphs and for weak selection, the emergence of cooperation can be assessed by structure coefficients, which can be specified for each configuration and each regular graph. Thus, as a single cooperator can be interpreted as a lone mutant, the configuration-based structure coefficients also describe fixation properties of multiple mutants. We analyze the structure coefficients and particularly show that under certain conditions, the coefficients strongly correlate to the average shortest path length between cooperators on the evolutionary graph. Thus, for multiple cooperators fixation properties on regular evolutionary graphs can be linked to cooperator path lengths.

Environmental fitness heterogeneity in the Moran process

Many mathematical models of evolution assume that all individuals experience the same environment. Here, we study the Moran process in heterogeneous environments. The population is of finite size with two competing types, which are exposed to a fixed number of environmental conditions. Reproductive rate is determined by both the type and the environment. We first calculate the condition for selection to favour the mutant relative to the resident wild-type. In large populations, the mutant is favoured if and only if the mutant's spatial average reproductive rate exceeds that of the resident. But environmental heterogeneity elucidates an interesting asymmetry between the mutant and the resident. Specifically, mutant heterogeneity suppresses its fixation probability; if this heterogeneity is strong enough, it can even completely offset the effects of selection (including in large populations). By contrast, resident heterogeneity has no effect on a mutant's fixation probability in large populations and can amplify it in small populations.

A mathematical formalism for natural selection with arbitrary spatial and genetic structure

We define a general class of models representing natural selection between two alleles. The population size and spatial structure are arbitrary, but fixed. Genetics can be haploid, diploid, or otherwise; reproduction can be asexual or sexual. Biological events (e.g. births, deaths, mating, dispersal) depend in arbitrary fashion on the current population state. Our formalism is based on the idea of genetic sites. Each genetic site resides at a particular locus and houses a single allele. Each individual contains a number of sites equal to its ploidy (one for haploids, two for diploids, etc.). Selection occurs via replacement events, in which alleles in some sites are replaced by copies of others. Replacement events depend stochastically on the population state, leading to a Markov chain representation of natural selection. Within this formalism, we define reproductive value, fitness, neutral drift, and fixation probability, and prove relationships among them. We identify four criteria for evaluating which allele is selected and show that these become equivalent in the limit of low mutation. We then formalize the method of weak selection. The power of our formalism is illustrated with applications to evolutionary games on graphs and to selection in a haplodiploid population.

Stationary frequencies and mixing times for neutral drift processes with spatial structure

We study a general setting of neutral evolution in which the population is of finite, constant size and can have spatial structure. Mutation leads to different genetic types (traits), which can be discrete or continuous. Under minimal assumptions, we show that the marginal trait distributions of the evolutionary process, which specify the probability that any given individual has a certain trait, all converge to the stationary distribution of the mutation process. In particular, the stationary frequencies of traits in the population are independent of its size, spatial structure and evolutionary update rule, and these frequencies can be calculated by evaluating a simple stochastic process describing a population of size one (i.e. the mutation process itself). We conclude by analysing mixing times, which characterize rates of convergence of the mutation process along the lineages, in terms of demographic variables of the evolutionary process.

Individualised aspiration dynamics: Calculation by proofs

Cooperation is key for the evolution of biological systems ranging from bacteria communities to human societies. Evolutionary processes can dramatically alter the cooperation level. Evolutionary processes are typically of two classes: comparison based and self-evaluation based. The fate of cooperation is extremely sensitive to the details of comparison based processes. For self-evaluation processes, however, it is still unclear whether the sensitivity remains. We concentrate on a class of self-evaluation processes based on aspiration, where all the individuals adjust behaviors based on their own aspirations. We prove that the evolutionary outcome with heterogeneous aspirations is the same as that of the homogeneous one for regular networks under weak selection limit. Simulation results further suggest that it is also valid for general networks across various distributions of personalised aspirations. Our result clearly indicates that self-evaluation processes are robust in contrast with comparison based rules. In addition, our result greatly simplifies the calculation of the aspiration dynamics, which is computationally expensive.

Cooperation is the cornerstone to understand how biological systems evolve. Previous studies have shown that cooperation is sensitive to the details of evolutionary processes, even if all the individuals update strategies in the same way. Here we propose a class of updating rules driven by self-evaluation, where each individual has its personal aspiration. The evolutionary outcome is the same as if all the individuals adopt the same aspiration for regular networks, provided the selection intensity is weak enough. In addition, we provide a simple numerical method to identify the favored strategy. Our result shows a very robust class of strategy updating rules. And it implies that complexity in updating rules does not necessarily lead to the sensitivity of evolutionary outcomes.

Population Structure Promotes the Evolution of Intuitive Cooperation and Inhibits Deliberation

Spatial structure is one of the most studied mechanisms in evolutionary game theory. Here, we explore the consequences of spatial structure for a question which has received considerable empirical and theoretical attention in recent years, but has not yet been studied from a network perspective: whether cooperation relies on intuitive predispositions or deliberative self-control. We examine this question using a model which integrates the “dual-process” framework from cognitive science with evolutionary game theory, and considers the evolution of agents who are embedded within a social network and only interact with their neighbors. In line with past work in well-mixed populations, we find that selection favors either the intuitive defector strategy which never deliberates, or the dual-process cooperator strategy which intuitively cooperates but uses deliberation to switch to defection when doing so is payoff-maximizing. We find that sparser networks (i.e., smaller average degree) facilitate the success of dual-process cooperators over intuitive defectors, while also reducing the level of deliberation that dual-process cooperators engage in; and that these results generalize across different kinds of networks. These observations demonstrate the important role that spatial structure can have not just on the evolution of cooperation, but on the co-evolution of cooperation and cognition.