Phase diagrams for three-strategy evolutionary prisoner's dilemma games on regular graphs

Evolution of reciprocity with limited payoff memory

Direct reciprocity is a mechanism for the evolution of cooperation in repeated social interactions. According to the literature, individuals naturally learn to adopt conditionally cooperative strategies if they have multiple encounters with their partner. Corresponding models have greatly facilitated our understanding of cooperation, yet they often make strong assumptions on how individuals remember and process payoff information. For example, when strategies are updated through social learning, it is commonly assumed that individuals compare their average payoffs. This would require them to compute (or remember) their payoffs against everyone else in the population. To understand how more realistic constraints influence direct reciprocity, we consider the evolution of conditional behaviours when individuals learn based on more recent experiences. Even in the most extreme case that they only take into account their very last interaction, we find that cooperation can still evolve. However, such individuals adopt less generous strategies, and they cooperate less often than in the classical setup with average payoffs. Interestingly, once individuals remember the payoffs of two or three recent interactions, cooperation rates quickly approach the classical limit. These findings contribute to a literature that explores which kind of cognitive capabilities are required for reciprocal cooperation. While our results suggest that some rudimentary form of payoff memory is necessary, it suffices to remember a few interactions.

Tax thresholds yield multiple optimal cooperation levels in the spatial public goods game

Income redistribution, which involves transferring income from certain individuals to others, plays a crucial role in human societies. Previous research has indicated that tax-based redistribution can promote cooperation by enhancing incentives for cooperators. In such a tax system, all individuals, irrespective of their income levels, contribute to the tax system, and the tax revenue is subsequently redistributed to everyone. In this study, we relax this assumption by introducing a tax threshold, signifying that only individuals with incomes exceeding the threshold will be subject to taxation. In particular, we employ the spatial public goods game to investigate the influence of tax rates-the percentage of income allocated to tax-and tax thresholds, which determine the income level at which individuals become taxable, on the evolution of cooperation. Our extensive numerical simulations disclose that tax thresholds produce complex outcomes for the evolution of cooperation, depending on tax rates. Notably, at low tax rates (i.e., below 0.41), as the tax threshold increases, discontinuous phase transitions in cooperation performance suggest the presence of multiple intervals of effective tax thresholds that promote peak cooperation levels. Nevertheless, irrespective of the chosen tax rate, once the tax threshold surpasses a critical threshold, the redistribution mechanism fails, causing the collapse of cooperation. Evolutionary snapshots show that self-organized redistribution forms an intermediary layer on the peripheries of cooperative clusters, effectively shielding cooperators from potential defectors. Quantitative analyses shed light on how self-organized redistribution narrows the income gap between cooperators and defectors through precise identification of tax-exempt entities, thereby amplifying the cooperative advantage. Collectively, these findings enhance our comprehension of how income redistribution influences cooperation, highlighting the pivotal role of tax thresholds.

The evolutionary extortion game of multiple groups in hypernetworks

As a type of zero-determinant strategies, the extortion strategy was found to be an evolutionarily stable strategy in structural groups. However, instead of complex networks structure, this paper focus on a multi-group game in hypernetworks, using the framework of a gift giving game driven by replicator-like dynamics. We find that the extortion is evolutionarily stable in the hypernetwork structure. The extortion game in hypernetworks can promote the emergence of the cooperative behavior compared to the traditional dual-strategy game and the extortion game in complex networks. The results show that the cooperation behavior attracts most of the groups for the smaller benefit value. With the increase of benefit value, cooperators turn into defectors and extortioners, but cooperation behavior still survives in hypernetworks under extreme conditions. Moreover, small-scale groups are more conducive to cooperation.

Cooperation in alternating interactions with memory constraints

In repeated social interactions, individuals often employ reciprocal strategies to maintain cooperation. To explore the emergence of reciprocity, many theoretical models assume synchronized decision making. In each round, individuals decide simultaneously whether to cooperate or not. Yet many manifestations of reciprocity in nature are asynchronous. Individuals provide help at one time and receive help at another. Here, we explore such alternating games in which players take turns. We mathematically characterize all Nash equilibria among memory-one strategies. Moreover, we use evolutionary simulations to explore various model extensions, exploring the effect of discounted games, irregular alternation patterns, and higher memory. In all cases, we observe that mutual cooperation still evolves for a wide range of parameter values. However, compared to simultaneous games, alternating games require different strategies to maintain cooperation in noisy environments. Moreover, none of the respective strategies are evolutionarily stable.

In many instances of reciprocity, individuals cooperate in turns. Here, the authors analyze the equilibria and the dynamics of such alternating games, and in particular describe all strategies with one-round memory that maintain cooperation.

Memory-two zero-determinant strategies in repeated games

Repeated games have provided an explanation of how mutual cooperation can be achieved even if defection is more favourable in a one-shot game in the Prisoner's Dilemma situation. Recently found zero-determinant (ZD) strategies have substantially been investigated in evolutionary game theory. The original memory-one ZD strategies unilaterally enforce linear relationships between average pay-offs of players. Here, we extend the concept of ZD strategies to memory-two strategies in repeated games. Memory-two ZD strategies unilaterally enforce linear relationships between correlation functions of pay-offs and pay-offs of the previous round. Examples of memory-two ZD strategy in the repeated Prisoner's Dilemma game are provided, some of which generalize the tit-for-tat strategy to a memory-two case. Extension of ZD strategies to memory-n case with n ≥ ~2 is also straightforward.

Automata representation of successful strategies for social dilemmas

In a social dilemma, cooperation is collectively optimal, yet individually each group member prefers to defect. A class of successful strategies of direct reciprocity were recently found for the iterated prisoner’s dilemma and for the iterated three-person public-goods game: By a successful strategy, we mean that it constitutes a cooperative Nash equilibrium under implementation error, with assuring that the long-term payoff never becomes less than the co-players’ regardless of their strategies, when the error rate is small. Although we have a list of actions prescribed by each successful strategy, the rationale behind them has not been fully understood for the iterated public-goods game because the list has hundreds of entries to deal with every relevant history of previous interactions. In this paper, we propose a method to convert such history-based representation into an automaton with a minimal number of states. Our main finding is that a successful strategy for the iterated three-person public-goods game can be represented as a 10-state automaton by this method. In this automaton, each state can be interpreted as the player’s internal judgement of the situation, such as trustworthiness of the co-players and the need to redeem oneself after defection. This result thus suggests a comprehensible way to choose an appropriate action at each step towards cooperation based on a situational judgement, which is mapped from the history of interactions.

Caring for the future can turn tragedy into comedy for long-term collective action under risk of collapse

We will need collective action to avoid catastrophic climate change, and this will require valuing the long term as well as the short term. Shortsightedness and uncertainty have hindered progress in resolving this collective action problem and have been recognized as important barriers to cooperation among humans. Here, we propose a coupled social-ecological dilemma to investigate the interdependence of three well-identified components of this cooperation problem: 1) timescales of collapse and recovery in relation to time preferences regarding future outcomes, 2) the magnitude of the impact of collapse, and 3) the number of actors in the collective. We find that, under a sufficiently severe and time-distant collapse, how much the actors care for the future can transform the game from a tragedy of the commons into one of coordination, and even into a comedy of the commons in which cooperation dominates. Conversely, we also find conditions under which even strong concern for the future still does not transform the problem from tragedy to comedy. For a large number of participating actors, we find that the critical collapse impact, at which these game regime changes happen, converges to a fixed value of collapse impact per actor that is independent of the enhancement factor of the public good, which is usually regarded as the driver of the dilemma. Our results not only call for experimental testing but also help explain why polarization in beliefs about human-caused climate change can threaten global cooperation agreements.

A novel route to cyclic dominance in voluntary social dilemmas

Cooperation is the backbone of modern human societies, making it a priority to understand how successful cooperation-sustaining mechanisms operate. Cyclic dominance, a non-transitive set-up comprising at least three strategies wherein the first strategy overrules the second, which overrules the third, which, in turn, overrules the first strategy, is known to maintain biodiversity, drive competition between bacterial strains, and preserve cooperation in social dilemmas. Here, we present a novel route to cyclic dominance in voluntary social dilemmas by adding to the traditional mix of cooperators, defectors and loners, a fourth player type, risk-averse hedgers, who enact tit-for-tat upon paying a hedging cost to avoid being exploited. When this cost is sufficiently small, cooperators, defectors and hedgers enter a loop of cyclic dominance that preserves cooperation even under the most adverse conditions. By contrast, when the hedging cost is large, hedgers disappear, consequently reverting to the traditional interplay of cooperators, defectors, and loners. In the interim region of hedging costs, complex evolutionary dynamics ensues, prompting transitions between states with two, three or four competing strategies. Our results thus reveal that voluntary participation is but one pathway to sustained cooperation via cyclic dominance.

The influence of heterogeneous learning ability on the evolution of cooperation

In this paper, we design a simple coevolution model to investigate the role of heterogeneous learning ability on the evolution of cooperation. The model weakens the winner's learning ability in order to keep its current advantage. Conversely, it strengthens the loser's learning ability for increasing the chance to update its strategy. In particular, we consider this coevolutionary model separately applying to both cooperators and defectors (rule I), only cooperators (rule II), as well as only defectors (rule III) in spatial prisoner's dilemma game. Through numerical simulations, we find that cooperation can be promoted in rule II, whereas, cooperation is hampered in rule I and rule III. We reveal its potential reason from the viewpoint of enduring and expanding periods in game dynamics. Our results thus provide a deeper understanding regarding the heterogeneous learning ability on game theory.

Thermodynamics of evolutionary games

How cooperation can evolve between players is an unsolved problem of biology. Here we use Hamiltonian dynamics of models of the Ising type to describe populations of cooperating and defecting players to show that the equilibrium fraction of cooperators is given by the expectation value of a thermal observable akin to a magnetization. We apply the formalism to the public goods game with three players and show that a phase transition between cooperation and defection occurs that is equivalent to a transition in one-dimensional Ising crystals with long-range interactions. We then investigate the effect of punishment on cooperation and find that punishment plays the role of a magnetic field that leads to an "alignment" between players, thus encouraging cooperation. We suggest that a thermal Hamiltonian picture of the evolution of cooperation can generate other insights about the dynamics of evolving groups by mining the rich literature of critical dynamics in low-dimensional spin systems.

Evolution of cooperation in stochastic games

Social dilemmas occur when incentives for individuals are misaligned with group interests^1-7. According to the 'tragedy of the commons', these misalignments can lead to overexploitation and collapse of public resources. The resulting behaviours can be analysed with the tools of game theory⁸. The theory of direct reciprocity^9-15 suggests that repeated interactions can alleviate such dilemmas, but previous work has assumed that the public resource remains constant over time. Here we introduce the idea that the public resource is instead changeable and depends on the strategic choices of individuals. An intuitive scenario is that cooperation increases the public resource, whereas defection decreases it. Thus, cooperation allows the possibility of playing a more valuable game with higher payoffs, whereas defection leads to a less valuable game. We analyse this idea using the theory of stochastic games^16-19 and evolutionary game theory. We find that the dependence of the public resource on previous interactions can greatly enhance the propensity for cooperation. For these results, the interaction between reciprocity and payoff feedback is crucial: neither repeated interactions in a constant environment nor single interactions in a changing environment yield similar cooperation rates. Our framework shows which feedbacks between exploitation and environment-either naturally occurring or designed-help to overcome social dilemmas.

Crosstalk in concurrent repeated games impedes direct reciprocity and requires stronger levels of forgiveness

Direct reciprocity is a mechanism for cooperation among humans. Many of our daily interactions are repeated. We interact repeatedly with our family, friends, colleagues, members of the local and even global community. In the theory of repeated games, it is a tacit assumption that the various games that a person plays simultaneously have no effect on each other. Here we introduce a general framework that allows us to analyze “crosstalk” between a player’s concurrent games. In the presence of crosstalk, the action a person experiences in one game can alter the person’s decision in another. We find that crosstalk impedes the maintenance of cooperation and requires stronger levels of forgiveness. The magnitude of the effect depends on the population structure. In more densely connected social groups, crosstalk has a stronger effect. A harsh retaliator, such as Tit-for-Tat, is unable to counteract crosstalk. The crosstalk framework provides a unified interpretation of direct and upstream reciprocity in the context of repeated games.

Social interactions among people are often repeated, and yet it is assumed that simultaneous interactions are independent from one another. Here, Reiter and colleagues describe a conceptual framework where an action in one game can influence the decision in another.

Memory-n strategies of direct reciprocity

Humans routinely use conditionally cooperative strategies when interacting in repeated social dilemmas. They are more likely to cooperate if others cooperated before, and are ready to retaliate if others defected. To capture the emergence of reciprocity, most previous models consider subjects who can only choose from a restricted set of representative strategies, or who react to the outcome of the very last round only. As players memorize more rounds, the dimension of the strategy space increases exponentially. This increasing computational complexity renders simulations for individuals with higher cognitive abilities infeasible, especially if multiplayer interactions are taken into account. Here, we take an axiomatic approach instead. We propose several properties that a robust cooperative strategy for a repeated multiplayer dilemma should have. These properties naturally lead to a unique class of cooperative strategies, which contains the classical Win-Stay Lose-Shift rule as a special case. A comprehensive numerical analysis for the prisoner's dilemma and for the public goods game suggests that strategies of this class readily evolve across various memory-n spaces. Our results reveal that successful strategies depend not only on how cooperative others were in the past but also on the respective context of cooperation.

Benefits of tolerance in public goods games

Leaving the joint enterprise when defection is unveiled is always a viable option to avoid being exploited. Although loner strategy helps the population not to be trapped into the tragedy of the commons state, it could offer only a modest income for nonparticipants. In this paper we demonstrate that showing some tolerance toward defectors could not only save cooperation in harsh environments but in fact results in a surprisingly high average payoff for group members in public goods games. Phase diagrams and the underlying spatial patterns reveal the high complexity of evolving states where cyclic dominant strategies or two-strategy alliances can characterize the final state of evolution. We identify microscopic mechanisms which are responsible for the superiority of global solutions containing tolerant players. This phenomenon is robust and can be observed both in well-mixed and in structured populations highlighting the importance of tolerance in our everyday life.

Nonlinear dynamics of the rock-paper-scissors game with mutations

We analyze the replicator-mutator equations for the rock-paper-scissors game. Various graph-theoretic patterns of mutation are considered, ranging from a single unidirectional mutation pathway between two of the species, to global bidirectional mutation among all the species. Our main result is that the coexistence state, in which all three species exist in equilibrium, can be destabilized by arbitrarily small mutation rates. After it loses stability, the coexistence state gives birth to a stable limit cycle solution created in a supercritical Hopf bifurcation. This attracting periodic solution exists for all the mutation patterns considered, and persists arbitrarily close to the limit of zero mutation rate and a zero-sum game.

Peer pressure: enhancement of cooperation through mutual punishment

An open problem in evolutionary game dynamics is to understand the effect of peer pressure on cooperation in a quantitative manner. Peer pressure can be modeled by punishment, which has been proved to be an effective mechanism to sustain cooperation among selfish individuals. We investigate a symmetric punishment strategy, in which an individual will punish each neighbor if their strategies are different, and vice versa. Because of the symmetry in imposing the punishment, one might intuitively expect the strategy to have little effect on cooperation. Utilizing the prisoner's dilemma game as a prototypical model of interactions at the individual level, we find, through simulation and theoretical analysis, that proper punishment, when even symmetrically imposed on individuals, can enhance cooperation. Also, we find that the initial density of cooperators plays an important role in the evolution of cooperation driven by mutual punishment.

Cyclic dominance in evolutionary games: a review

Rock is wrapped by paper, paper is cut by scissors and scissors are crushed by rock. This simple game is popular among children and adults to decide on trivial disputes that have no obvious winner, but cyclic dominance is also at the heart of predator-prey interactions, the mating strategy of side-blotched lizards, the overgrowth of marine sessile organisms and competition in microbial populations. Cyclical interactions also emerge spontaneously in evolutionary games entailing volunteering, reward, punishment, and in fact are common when the competing strategies are three or more, regardless of the particularities of the game. Here, we review recent advances on the rock-paper-scissors (RPS) and related evolutionary games, focusing, in particular, on pattern formation, the impact of mobility and the spontaneous emergence of cyclic dominance. We also review mean-field and zero-dimensional RPS models and the application of the complex Ginzburg-Landau equation, and we highlight the importance and usefulness of statistical physics for the successful study of large-scale ecological systems. Directions for future research, related, for example, to dynamical effects of coevolutionary rules and invasion reversals owing to multi-point interactions, are also outlined.

Evolutionary stability in the asymmetric volunteer's dilemma

It is often assumed that in public goods games, contributors are either strong or weak players and each individual has an equal probability of exhibiting cooperation. It is difficult to explain why the public good is produced by strong individuals in some cooperation systems, and by weak individuals in others. Viewing the asymmetric volunteer's dilemma game as an evolutionary game, we find that whether the strong or the weak players produce the public good depends on the initial condition (i.e., phenotype or initial strategy of individuals). These different evolutionarily stable strategies (ESS) associated with different initial conditions, can be interpreted as the production modes of public goods of different cooperation systems. A further analysis revealed that the strong player adopts a pure strategy but mixed strategies for the weak players to produce the public good, and that the probability of volunteering by weak players decreases with increasing group size or decreasing cost-benefit ratio. Our model shows that the defection probability of a "strong" player is greater than the "weak" players in the model of Diekmann (1993). This contradicts Selten's (1980) model that public goods can only be produced by a strong player, is not an evolutionarily stable strategy, and will therefore disappear over evolutionary time. Our public good model with ESS has thus extended previous interpretations that the public good can only be produced by strong players in an asymmetric game.

The role of opportunistic migration in cyclic games

We study cyclic evolutionary games in a spatial diluted grid environment in which agents strategically interact locally but can also opportunistically move to other positions within a given migration radius. We find that opportunistic migration can inverse the cyclic prevalence between the strategies when the frequency of random imitation is large enough compared to the payoff-driven imitation. At the transition the average size of the patterns diverges and this threatens diversity of strategies.

Aspiration-based partner switching boosts cooperation in social dilemmas

Most previous studies concerning linking dynamics often assumed that links pairing individuals should be identified and treated differently during topology adjusting procedure, in order to promote cooperation. A common assumption was that cooperators were expected to avoid being exploited by quickly breaking up relationships with defectors. Then the so-called prosocial links linking two cooperators (abbreviated as CC links hereafter) would be much favored by evolution, whereby cooperation was promoted. However, we suggest that this is not always necessary. Here, we developed a minimal model in which an aspiration-based partner switching mechanism was embedded to regulate the evolution of cooperation in social dilemmas. Individuals adjusted social ties in a self-questioning manner in line with the learning theory. Less game information was involved during dynamic linking and all links were tackled anonymously irrespective of their types (i.e., CD links, DD links, or CC links). The main results indicate that cooperation flourishes for a broad range of parameters. The denser the underlying network, the more difficult the evolution of cooperation. More importantly, moderate aspirations do much better in promoting the evolution of altruistic behavior and for most cases there exists the optimal aspiration level that most benefits cooperation. Too strong or too weak selection intensity turns out to be pretty conducive to the evolution of cooperation in such a dynamical system.

Stochastic heterogeneous interaction promotes cooperation in spatial prisoner's dilemma game

Previous studies mostly investigate player's cooperative behavior as affected by game time-scale or individual diversity. In this paper, by involving both time-scale and diversity simultaneously, we explore the effect of stochastic heterogeneous interaction. In our model, the occurrence of game interaction between each pair of linked player obeys a random probability, which is further described by certain distributions. Simulations on a 4-neighbor square lattice show that the cooperation level is remarkably promoted when stochastic heterogeneous interaction is considered. The results are then explained by investigating the mean payoffs, the mean boundary payoffs and the transition probabilities between cooperators and defectors. We also show some typical snapshots and evolution time series of the system. Finally, the 8-neighbor square lattice and BA scale-free network results indicate that the stochastic heterogeneous interaction can be robust against different network topologies. Our work may sharpen the understanding of the joint effect of game time-scale and individual diversity on spatial games.

Evolution of extortion in structured populations

Extortion strategies can dominate any opponent in an iterated prisoner's dilemma game. But if players are able to adopt the strategies performing better, extortion becomes widespread and evolutionary unstable. It may sometimes act as a catalyst for the evolution of cooperation, and it can also emerge in interactions between two populations, yet it is not the evolutionarily stable outcome. Here we revisit these results in the realm of spatial games. We find that pairwise imitation and birth-death dynamics return known evolutionary outcomes. Myopic best response strategy updating, on the other hand, reveals counterintuitive solutions. Defectors and extortioners coarsen spontaneously, which allows cooperators to prevail even at prohibitively high temptations to defect. Here extortion strategies play the role of a Trojan horse. They may emerge among defectors by chance, and once they do, cooperators become viable as well. These results are independent of the interaction topology, and they highlight the importance of coarsening, checkerboard ordering, and best response updating in evolutionary games.

Impact of social punishment on cooperative behavior in complex networks

Social punishment is a mechanism by which cooperative individuals spend part of their resources to penalize defectors. In this paper, we study the evolution of cooperation in 2-person evolutionary games on networks when a mechanism for social punishment is introduced. Specifically, we introduce a new kind of role, punisher, which is aimed at reducing the earnings of defectors by applying to them a social fee. Results from numerical simulations show that different equilibria allowing the three strategies to coexist are possible as well as that social punishment further enhance the robustness of cooperation. Our results are confirmed for different network topologies and two evolutionary games. In addition, we analyze the microscopic mechanisms that give rise to the observed macroscopic behaviors in both homogeneous and heterogeneous networks. Our conclusions might provide additional insights for understanding the roots of cooperation in social systems.

Evolution of cooperation in spatial traveler's dilemma game

Traveler's dilemma (TD) is one of social dilemmas which has been well studied in the economics community, but it is attracted little attention in the physics community. The TD game is a two-person game. Each player can select an integer value between and () as a pure strategy. If both of them select the same value, the payoff to them will be that value. If the players select different values, say and (), then the payoff to the player who chooses the small value will be and the payoff to the other player will be . We term the player who selects a large value as the cooperator, and the one who chooses a small value as the defector. The reason is that if both of them select large values, it will result in a large total payoff. The Nash equilibrium of the TD game is to choose the smallest value . However, in previous behavioral studies, players in TD game typically select values that are much larger than , and the average selected value exhibits an inverse relationship with . To explain such anomalous behavior, in this paper, we study the evolution of cooperation in spatial traveler's dilemma game where the players are located on a square lattice and each player plays TD games with his neighbors. Players in our model can adopt their neighbors' strategies following two standard models of spatial game dynamics. Monte-Carlo simulation is applied to our model, and the results show that the cooperation level of the system, which is proportional to the average value of the strategies, decreases with increasing until is greater than the critical value where cooperation vanishes. Our findings indicate that spatial reciprocity promotes the evolution of cooperation in TD game and the spatial TD game model can interpret the anomalous behavior observed in previous behavioral experiments.

Phase diagrams for the spatial public goods game with pool punishment

The efficiency of institutionalized punishment is studied by evaluating the stationary states in the spatial public goods game comprising unconditional defectors, cooperators, and cooperating pool punishers as the three competing strategies. Fines and costs of pool punishment are considered as the two main parameters determining the stationary distributions of strategies on the square lattice. Each player collects a payoff from five five-person public goods games, and the evolution of strategies is subsequently governed by imitation based on pairwise comparisons at a low level of noise. The impact of pool punishment on the evolution of cooperation in structured populations is significantly different from that reported previously for peer punishment. Representative phase diagrams reveal remarkably rich behavior, depending also on the value of the synergy factor that characterizes the efficiency of investments payed into the common pool. Besides traditional single- and two-strategy stationary states, a rock-paper-scissors type of cyclic dominance can emerge in strikingly different ways.

Cyclic competition of mobile species on continuous space: pattern formation and coexistence

We propose a model for cyclically competing species on continuous space and investigate the effect of the interplay between the interaction range and mobility on coexistence. A transition from coexistence to extinction is uncovered with a strikingly nonmonotonic behavior in the coexistence probability. About the minimum in the probability, switches between spiral and plane-wave patterns arise. A strong mobility can either promote or hamper coexistence, depending on the radius of the interaction range. These phenomena are absent in any lattice-based model, and we demonstrate that they can be explained using nonlinear partial differential equations. Our continuous-space model is more physical and we expect the findings to generate experimental interest.

Role of aspiration-induced migration in cooperation

Both cooperation and migration are ubiquitous in human society and animal world. In this Rapid Communication, we propose an aspiration-induced migration in which individuals will migrate to new sites provided that their payoffs are below some aspiration level. It is found that moderate aspiration level can best favor cooperative behavior. In particular, moderate aspiration level enables cooperator clusters to maintain and expand whereas induces defector clusters to disintegrate, thus promoting the diffusion of cooperation among population. Our results provide insights into understanding the role played by migration in the emergence of cooperative behavior.