Evolutionary performance of zero-determinant strategies in multiplayer games

Mitigation of block withholding attack based on zero-determinant strategy

This article focuses on the mining dilemma of block withholding attack between the mining pools in the bitcoin system. In order to obtain the higher revenue, the rational mining pool usually chooses an infiltration attack, that is, the pool will falls into the mining dilemma of the PoW consensus algorithm. Thus the article proposes to apply zero-determinant strategies for optimizing the behavior selection of the mining pool under PoW consensus mechanism to increase the total revenues of the system, so as to solve the mining dilemma. After theoretically studying the set and extortionate strategy of zero-determinant, the article devises an adaptive zero-determinant strategy that the pool can change the corporation probability of the next round based on its previous revenues. To verify the effectiveness of zero-determinant strategies, based on the actual revenue of the mining pool defined and deduced in the paper, it simulates 30 sets of game strategies to illustrate the revenue variation of the mining pools. The simulation results show that the three zero-determinant strategies can effectively improve the convergence rate of cooperation, mitigate block withholding attack and maximize the total revenues of the system. Compared with the set and extortionate strategy, the adaptive strategy can ensure more stability and more revenue.

Adapting paths against zero-determinant strategies in repeated prisoner's dilemma games

Long-term cooperation, competition, or exploitation among individuals can be modeled through repeated games. In repeated games, Press and Dyson discovered zero-determinant (ZD) strategies that enforce a special relationship between two players. This special relationship implies that a ZD player can unilaterally impose a linear payoff relationship to the opponent regardless of the opponent's strategies. A ZD player also has a property that can lead the opponent to an unconditional cooperation if the opponent tries to improve its payoff. This property has been mathematically confirmed by Chen and Zinger. Humans often underestimate a payoff obtained in the future. However, such discounting was not considered in their analysis. Here, we mathematically explored whether a ZD player can lead the opponent to an unconditional cooperation even if a discount factor is incorporated. Specifically, we represented the expected payoff with a discount factor as the form of determinants and calculated whether the values obtained by partially differentiating each factor in the strategy vector become positive. As a result, we proved that the strategy vector ends up as an unconditional cooperation even when starting from any initial strategy. This result was confirmed through numerical calculations. We extended the applicability of ZD strategies to real world problems.

Evolutionary dynamics of zero-determinant strategies in repeated multiplayer games

Several studies have confirmed the existence of zero-determinant (ZD) strategies in repeated social dilemmas since Press and Dyson's ingenious discovery of ZD strategies in iterated prisoner's dilemmas. However, less research studies evolutionary performance of multiplayer ZD strategies, especially from a theoretical perspective. Here, we use a state-clustering method to theoretically analyze evolutionary dynamics of two representative ZD strategies: generous ZD strategies and extortionate ZD strategies. We consider two new settings for multiplayer ZD strategies: competitions with all ZD strategies and competitions with all memory-one strategies, apart from the competitions between these strategies and some classical ones. Moreover, we investigate the influence of the level of generosity and extortion on evolutionary dynamics of generous and extortionate ZD strategies, which was commonly ignored in previous studies. Theoretical results show that players with limited generosity are at an advantageous place and extortioners extorting more severely hold their ground more readily. Our results may provide new insights into better understanding evolutionary dynamics of ZD strategies in repeated multiplayer games.

The Evolution of Cooperation in Two-Dimensional Mobile Populations with Random and Strategic Dispersal

We investigate the effect of the environment dimensionality and different dispersal strategies on the evolution of cooperation in a finite structured population of mobile individuals. We consider a population consisting of cooperators and free-riders residing on a two-dimensional lattice with periodic boundaries. Individuals explore the environment according to one of the four dispersal strategies and interact with each other via a public goods game. The population evolves according to a birth–death–birth process with the fitness of the individuals deriving from the game-induced payouts. We found that the outcomes of the strategic dispersal strategies in the two-dimensional setting are identical to the outcomes in the one-dimensional setting. The random dispersal strategy, not surprisingly, resulted in the worst outcome for cooperators.

Payoff landscapes and the robustness of selfish optimization in iterated games

In iterated games, a player can unilaterally exert influence over the outcome through a careful choice of strategy. A powerful class of such "payoff control" strategies was discovered by Press and Dyson (2012). Their so-called "zero-determinant" (ZD) strategies allow a player to unilaterally enforce a linear relationship between both players' payoffs. It was subsequently shown by Chen and Zinger (2014) that when the slope of this linear relationship is positive, ZD strategies are robustly effective against a selfishly optimizing co-player, in that all adapting paths of the selfish player lead to the maximal payoffs for both players (at least when there are certain restrictions on the game parameters). In this paper, we investigate the efficacy of selfish learning against a fixed player in more general settings, for both ZD and non-ZD strategies. We first prove that in any symmetric 2[Formula: see text]2 game, the selfish player's final strategy must be of a certain form and cannot be fully stochastic. We then show that there are prisoner's dilemma interactions for which selfish optimization does not always lead to maximal payoffs against fixed ZD strategies with positive slope. We give examples of selfish adapting paths that lead to locally but not globally optimal payoffs, undermining the robustness of payoff control strategies. For non-ZD strategies, these pathologies arise regardless of the original restrictions on the game parameters. Our results illuminate the difficulty of implementing robust payoff control and selfish optimization, even in the simplest context of playing against a fixed strategy.

Conditions for the existence of zero-determinant strategies under observation errors in repeated games

Repeated games are useful models to analyze long term interactions of living species and complex social phenomena. Zero-determinant (ZD) strategies in repeated games discovered by Press and Dyson in 2012 enforce a linear payoff relationship between a focal player and the opponent. This linear relationship can be set arbitrarily by a ZD player. Hence, a subclass of ZD strategies can fix the opponent's expected payoff and another subclass of the strategies can exceed the opponent for the expected payoff. Since this discovery, theories for ZD strategies are extended to cope with various natural situations. It is especially important to consider the theory of ZD strategies for repeated games with a discount factor and observation errors because it allows the theory to be applicable in the real world. Recent studies revealed their existence of ZD strategies even in repeated games with both factors. However, the conditions for the existence has not been sufficiently analyzed. Here, we mathematically analyzed the conditions in repeated games with both factors. First, we derived the thresholds of a discount factor and observation errors which ensure the existence of Equalizer and positively correlated ZD (pcZD) strategies, which are well-known subclasses of ZD strategies. We found that ZD strategies exist only when a discount factor remains high as the error rates increase. Next, we derived the conditions for the expected payoff of the opponent enforced by Equalizer as well as the conditions for the slope and base line payoff of linear lines enforced by pcZD. As a result, we found that, as error rates increase or a discount factor decreases, the conditions for the linear line that Equalizer or pcZD can enforce become strict.

Zero-determinant strategies under observation errors in repeated games

Zero-determinant (ZD) strategies are a novel class of strategies in the repeated prisoner's dilemma (RPD) game discovered by Press and Dyson. This strategy set enforces a linear payoff relationship between a focal player and the opponent regardless of the opponent's strategy. In the RPD game, games with discounting and observation errors represent an important generalization, because they are better able to capture real life interactions which are often noisy. However, they have not been considered in the original discovery of ZD strategies. In some preceding studies, each of them has been considered independently. Here, we analytically study the strategies that enforce linear payoff relationships in the RPD game considering both a discount factor and observation errors. As a result, we first reveal that the payoffs of two players can be represented by the form of determinants as shown by Press and Dyson even with the two factors. Then, we search for all possible strategies that enforce linear payoff relationships and find that both ZD strategies and unconditional strategies are the only strategy sets to satisfy the condition. We also show that neither Extortion nor Generous strategies, which are subsets of ZD strategies, exist when there are errors. Finally, we numerically derive the threshold values above which the subsets of ZD strategies exist. These results contribute to a deep understanding of ZD strategies in society.

Strategically influencing an uncertain future

Many of today's most pressing societal concerns require decisions which take into account a distant and uncertain future. Recent developments in strategic decision-making suggest that individuals, or a small group of individuals, can unilaterally influence the collective outcome of such complex social dilemmas. However, these results do not account for the extent to which decisions are moderated by uncertainty in the probability or timing of future outcomes that characterise the valuation of a (distant) uncertain future. Here we develop a general framework that captures interactions among uncertainty, the resulting time-inconsistent discounting, and their consequences for decision-making processes. In deterministic limits, existing theories can be recovered. More importantly, new insights are obtained into the possibilities for strategic influence when the valuation of the future is uncertain. We show that in order to unilaterally promote and sustain cooperation in social dilemmas, decisions of generous and extortionate strategies should be adjusted to the level of uncertainty. In particular, generous payoff relations cannot be enforced during periods of greater risk (which we term the "generosity gap"), unless the strategic enforcer orients their strategy towards a more distant future by consistently choosing "selfless" cooperative decisions; likewise, the possibilities for extortion are directly limited by the level of uncertainty. Our results have implications for policies that aim to solve societal concerns with consequences for a distant future and provides a theoretical starting point for investigating how collaborative decision-making can help solve long-standing societal dilemmas.

The evolution of universal adaptations of life is driven by universal properties of matter: energy, entropy, and interaction

The evolution of multicellular eukaryotes expresses two sorts of adaptations: local adaptations like fur or feathers, which characterize species in particular environments, and universal adaptations like microbiomes or sexual reproduction, which characterize most multicellulars in any environment. We reason that the mechanisms driving the universal adaptations of multicellulars should themselves be universal, and propose a mechanism based on properties of matter and systems: energy, entropy, and interaction. Energy from the sun, earth and beyond creates new arrangements and interactions. Metabolic networks channel some of this energy to form cooperating, interactive arrangements. Entropy, used here as a term for all forces that dismantle ordered structures (rather than as a physical quantity), acts as a selective force. Entropy selects for arrangements that resist it long enough to replicate, and dismantles those that do not. Interactions, energy-charged and dynamic, restrain entropy and enable survival and propagation of integrated living systems. This fosters survival-of-the-fitted - those entities that resist entropic destruction - and not only of the fittest - the entities with the greatest reproductive success. The "unit" of evolution is not a discrete entity, such as a gene, individual, or species; what evolves are collections of related interactions at multiple scales. Survival-of-the-fitted explains universal adaptations, including resident microbiomes, sexual reproduction, continuous diversification, programmed turnover, seemingly wasteful phenotypes, altruism, co-evolving environmental niches, and advancing complexity. Indeed survival-of-the-fittest may be a particular case of the survival-of-the-fitted mechanism, promoting local adaptations that express reproductive advantages in addition to resisting entropy. Survival-of-the-fitted accounts for phenomena that have been attributed to neutral evolution: in the face of entropy, there is no neutrality; all variations are challenged by ubiquitous energy and entropy, retaining those that are "fit enough". We propose experiments to test predictions of the survival-of-the-fitted theory, and discuss implications for the wellbeing of humans and the biosphere.

The evolution of universal adaptations of life is driven by universal properties of matter: energy, entropy, and interaction

The evolution of multicellular eukaryotes expresses two sorts of adaptations: local adaptations like fur or feathers, which characterize species in particular environments, and universal adaptations like microbiomes or sexual reproduction, which characterize most multicellulars in any environment. We reason that the mechanisms driving the universal adaptations of multicellulars should themselves be universal, and propose a mechanism based on properties of matter and systems: energy, entropy, and interaction. Energy from the sun, earth and beyond creates new arrangements and interactions. Metabolic networks channel some of this energy to form cooperating, interactive arrangements. Entropy, used here as a term for all forces that dismantle ordered structures (rather than as a physical quantity), acts as a selective force. Entropy selects for arrangements that resist it long enough to replicate, and dismantles those that do not. Interactions, energy-charged and dynamic, restrain entropy and enable survival and propagation of integrated living systems. This fosters survival-of-the-fitted - those entities that resist entropic destruction - and not only of the fittest - the entities with the greatest reproductive success. The "unit" of evolution is not a discrete entity, such as a gene, individual, or species; what evolves are collections of related interactions at multiple scales. Survival-of-the-fitted explains universal adaptations, including resident microbiomes, sexual reproduction, continuous diversification, programmed turnover, seemingly wasteful phenotypes, altruism, co-evolving environmental niches, and advancing complexity. Indeed survival-of-the-fittest may be a particular case of the survival-of-the-fitted mechanism, promoting local adaptations that express reproductive advantages in addition to resisting entropy. Survival-of-the-fitted accounts for phenomena that have been attributed to neutral evolution: in the face of entropy, there is no neutrality; all variations are challenged by ubiquitous energy and entropy, retaining those that are "fit enough". We propose experiments to test predictions of the survival-of-the-fitted theory, and discuss implications for the wellbeing of humans and the biosphere.

The evolution of universal adaptations of life is driven by universal properties of matter: energy, entropy, and interaction

The evolution of multicellular eukaryotes expresses two sorts of adaptations: local adaptations like fur or feathers, which characterize species in particular environments, and universal adaptations like microbiomes or sexual reproduction, which characterize most multicellulars in any environment. We reason that the mechanisms driving the universal adaptations of multicellulars should themselves be universal, and propose a mechanism based on properties of matter and systems: energy, entropy, and interaction. Energy from the sun, earth and beyond creates new arrangements and interactions. Metabolic networks channel some of this energy to form cooperating, interactive arrangements. Entropy, used here as a term for all forces that dismantle ordered structures (rather than as a physical quantity), acts as a selective force. Entropy selects for arrangements that resist it long enough to replicate, and dismantles those that do not. Interactions, energy-charged and dynamic, restrain entropy and enable survival and propagation of integrated living systems. This fosters survival-of-the-fitted - those entities that resist entropic destruction - and not only of the fittest - the entities with the greatest reproductive success. The "unit" of evolution is not a discrete entity, such as a gene, individual, or species; what evolves are collections of related interactions at multiple scales. Survival-of-the-fitted explains universal adaptations, including resident microbiomes, sexual reproduction, continuous diversification, programmed turnover, seemingly wasteful phenotypes, altruism, co-evolving environmental niches, and advancing complexity. Indeed survival-of-the-fittest may be a particular case of the survival-of-the-fitted mechanism, promoting local adaptations that express reproductive advantages in addition to resisting entropy. Survival-of-the-fitted accounts for phenomena that have been attributed to neutral evolution: in the face of entropy, there is no neutrality; all variations are challenged by ubiquitous energy and entropy, retaining those that are "fit enough". We propose experiments to test predictions of the survival-of-the-fitted theory, and discuss implications for the wellbeing of humans and the biosphere.

Cooperation dynamics of generalized reciprocity in state-based social dilemmas

We introduce a framework for studying social dilemmas in networked societies where individuals follow a simple state-based behavioral mechanism based on generalized reciprocity, which is rooted in the principle "help anyone if helped by someone." Within this general framework, which applies to a wide range of social dilemmas including, among others, public goods, donation, and snowdrift games, we study the cooperation dynamics on a variety of complex network examples. By interpreting the studied model through the lenses of nonlinear dynamical systems, we show that cooperation through generalized reciprocity always emerges as the unique attractor in which the overall level of cooperation is maximized, while simultaneously exploitation of the participating individuals is prevented. The analysis elucidates the role of the network structure, here captured by a local centrality measure which uniquely quantifies the propensity of the network structure to cooperation by dictating the degree of cooperation displayed both at the microscopic and macroscopic level. We demonstrate the applicability of the analysis on a practical example by considering an interaction structure that couples a donation process with a public goods game.

An Evolutionary Game-Theoretic Approach for Assessing Privacy Protection in mHealth Systems

With the rapid deployment of mobile technologies and their applications in the healthcare domain, privacy concerns have emerged as one of the most critical issues. Traditional technical and organizational approaches used to address privacy issues ignore economic factors, which are increasingly important in the investment strategy of those responsible for ensuring privacy protection. Taking the mHealth system as the context, this article builds an evolutionary game to model three types of entities (including system providers, hospitals and governments) under the conditions of incomplete information and bounded rationality. Given that the various participating entities are often unable to accurately estimate their own profits or costs, we propose a quantified approach to analyzing the optimal strategy of privacy investment and regulation. Numerical examples are provided for illustration and simulation purpose. Based upon these examples, several countermeasures and suggestions for privacy protection are proposed. Our analytical results show that governmental regulation and auditing has a significant impact on the strategic choice of the other two entities involved. In addition, the strategic choices of system providers and hospitals are not only correlated with profits and investment costs, but they are also significantly affected by free riding. If the profit growth coefficients increase to a critical level, mHealth system providers and hospitals will invest in privacy protection even without the imposition of regulations. However, the critical level is dependent on the values of the parameters (variables) in each case of investment and profits.

Towards an Iterated Game Model with Multiple Adversaries in Smart-World Systems

Diverse and varied cyber-attacks challenge the operation of the smart-world system that is supported by Internet-of-Things (IoT) (smart cities, smart grid, smart transportation, etc.) and must be carefully and thoughtfully addressed before widespread adoption of the smart-world system can be fully realized. Although a number of research efforts have been devoted to defending against these threats, a majority of existing schemes focus on the development of a specific defensive strategy to deal with specific, often singular threats. In this paper, we address the issue of coalitional attacks, which can be launched by multiple adversaries cooperatively against the smart-world system such as smart cities. Particularly, we propose a game-theory based model to capture the interaction among multiple adversaries, and quantify the capacity of the defender based on the extended Iterated Public Goods Game (IPGG) model. In the formalized game model, in each round of the attack, a participant can either cooperate by participating in the coalitional attack, or defect by standing aside. In our work, we consider the generic defensive strategy that has a probability to detect the coalitional attack. When the coalitional attack is detected, all participating adversaries are penalized. The expected payoff of each participant is derived through the equalizer strategy that provides participants with competitive benefits. The multiple adversaries with the collusive strategy are also considered. Via a combination of theoretical analysis and experimentation, our results show that no matter which strategies the adversaries choose (random strategy, win-stay-lose-shift strategy, or even the adaptive equalizer strategy), our formalized game model is capable of enabling the defender to greatly reduce the maximum value of the expected average payoff to the adversaries via provisioning sufficient defensive resources, which is reflected by setting a proper penalty factor against the adversaries. In addition, we extend our game model and analyze the extortion strategy, which can enable one participant to obtain more payoff by extorting his/her opponents. The evaluation results show that the defender can combat this strategy by encouraging competition among the adversaries, and significantly suppress the total payoff of the adversaries via setting the proper penalty factor.

Reciprocal Social Strategy in Social Repeated Games and Emergence of Social Norms

In an artificial society where agents repeatedly interact with one another, effective coordination among agents is generally a challenge. This is especially true when the participating agents are self-interested, and that there is no central authority to coordinate, and direct communication or negotiation are not possible. Recently, the problem was studied in a paper by Hao and Leung, where a new repeated game mechanism for modeling multi-agent interactions as well as a new reinforcement learning based agent learning method were proposed. In particular, the game mechanism differs from traditional repeated games in that the agents are anonymous, and the agents interact with randomly chosen opponents during each iteration. Their learning mechanism allows agents to coordinate without negotiations. The initial results had been promising. However, extended simulation also reveals that the outcomes are not stable in the long run in some cases, as the high level of cooperation is eventually not sustainable. In this work, we revisit he problem and propose a new learning mechanism as follows. First, we propose an enhanced Q-learning-based framework that allows the agents to better capture both the individual and social utilities that they have learned through observations. Second, we propose a new concept of \social attitude" for determining the action of the agents throughout the game. Simulation results reveal that this approach can achieve higher social utility, including close-to-optimal results in some scenarios, and more importantly, the results are sustainable with social norms emerging.

Updating Darwin: Information and entropy drive the evolution of life

The evolution of species, according to Darwin, is driven by struggle – by competition between variant autonomous individuals for survival of the fittest and reproductive advantage; the outcome of this struggle for survival is natural selection. The Neo-Darwinians reframed natural selection in terms of DNA: inherited genotypes directly encode expressed phenotypes; a fit phenotype means a fit genotype – thus the evolution of species is the evolution of selfish, reproducing individual genotypes. 

            Four general characteristics of advanced forms of life are not easily explained by this Neo-Darwinian paradigm: 1) Dependence on cooperation rather than on struggle, manifested by the microbiome, ecosystems and altruism; 2) The pursuit of diversity rather than optimal fitness, manifested by sexual reproduction; 3) Life’s investment in programmed death, rather then in open-ended survival; and 4) The acceleration of complexity, despite its intrinsic fragility.  

            Here I discuss two mechanisms that can resolve these paradoxical features; both mechanisms arise from viewing life as the evolution of information. Information has two inevitable outcomes; it increases by autocatalyis and it is destroyed by entropy. On the one hand, the autocalalysis of information inexorably drives the evolution of complexity, irrespective of its fragility. On the other hand, only those strategic arrangements that accommodate the destructive forces of entropy survive – cooperation, diversification, and programmed death result from the entropic selection of evolving species. Physical principles of information and entropy thus fashion the evolution of life.

Evolutionary consequences of behavioral diversity

Iterated games provide a framework to describe social interactions among groups of individuals. This body of work has focused primarily on individuals who face a simple binary choice, such as "cooperate" or "defect." Real individuals, however, can exhibit behavioral diversity, varying their input to a social interaction both qualitatively and quantitatively. Here we explore how access to a greater diversity of behavioral choices impacts the evolution of social dynamics in populations. We show that, in public goods games, some simple strategies that choose between only two possible actions can resist invasion by all multichoice invaders, even while engaging in relatively little punishment. More generally, access to a larger repertoire of behavioral choices results in a more "rugged" fitness landscape, with populations able to stabilize cooperation at multiple levels of investment. As a result, increased behavioral choice facilitates cooperation when returns on investments are low, but it hinders cooperation when returns on investments are high. Finally, we analyze iterated rock-paper-scissors games, the nontransitive payoff structure of which means that unilateral control is difficult to achieve. Despite this, we find that a large proportion of multichoice strategies can invade and resist invasion by single-choice strategies-so that even well-mixed populations will tend to evolve and maintain behavioral diversity.

Asymmetric Power Boosts Extortion in an Economic Experiment

Direct reciprocity is a major mechanism for the evolution of cooperation. Several classical studies have suggested that humans should quickly learn to adopt reciprocal strategies to establish mutual cooperation in repeated interactions. On the other hand, the recently discovered theory of ZD strategies has found that subjects who use extortionate strategies are able to exploit and subdue cooperators. Although such extortioners have been predicted to succeed in any population of adaptive opponents, theoretical follow-up studies questioned whether extortion can evolve in reality. However, most of these studies presumed that individuals have similar strategic possibilities and comparable outside options, whereas asymmetries are ubiquitous in real world applications. Here we show with a model and an economic experiment that extortionate strategies readily emerge once subjects differ in their strategic power. Our experiment combines a repeated social dilemma with asymmetric partner choice. In our main treatment there is one randomly chosen group member who is unilaterally allowed to exchange one of the other group members after every ten rounds of the social dilemma. We find that this asymmetric replacement opportunity generally promotes cooperation, but often the resulting payoff distribution reflects the underlying power structure. Almost half of the subjects in a better strategic position turn into extortioners, who quickly proceed to exploit their peers. By adapting their cooperation probabilities consistent with ZD theory, extortioners force their co-players to cooperate without being similarly cooperative themselves. Comparison to non-extortionate players under the same conditions indicates a substantial net gain to extortion. Our results thus highlight how power asymmetries can endanger mutually beneficial interactions, and transform them into exploitative relationships. In particular, our results indicate that the extortionate strategies predicted from ZD theory could play a more prominent role in our daily interactions than previously thought.

Unified and simple understanding for the evolution of conditional cooperators

Cooperation is a mysterious phenomenon which is observed in this world. The potential explanation is a repeated interaction. Cooperation is established if individuals meet the same opponent repeatedly and cooperate conditionally. Previous studies have analyzed the following four as characters of conditional cooperators mainly. (i) niceness (i.e., when a conditional cooperator meets an opponent in the first place, he (she) cooperates or defects), (ii) optimism (when a conditional cooperator meets an opponent in the past, but he (she) did not get access to information about the opponent's behavior in the previous round, he (she) cooperates or defects), (iii) generosity (even when a conditional cooperator knows that an opponent defected in the previous round, he (she) cooperates or defects) and (iv) retaliation (a conditional cooperator cooperates with a cooperator with a higher probability than with a defector). Previous works deal with these four characters mainly. However, these four characters basically have been regarded as distinct topics and unified understanding has not been done fully. Here we, by studying the iterated prisoner's dilemma game (in particular, additive games) and using evolutionarily stable strategy (ESS) analysis, find that when retaliation is large, the condition under which conditional cooperators are stable against the invasion by an unconditional defector is loose, while none of "niceness", "optimism", and "generosity" makes impact on the condition under which conditional cooperators are stable against an invasion by an unconditional defector. Furthermore, we show that we can understand "niceness", "optimism", and "generosity" uniformly by using one parameter indicating "cooperative", and when the conditional cooperators have large "retaliation" enough to resist an invasion by an unconditional defector, natural selection favors more "cooperative" conditional cooperators to invade the resident conditional cooperative strategy. Moreover, we show that these results are robust even when taking the existence of mistakes in behavior into consideration.

Evolutionary game theory using agent-based methods

Evolutionary game theory is a successful mathematical framework geared towards understanding the selective pressures that affect the evolution of the strategies of agents engaged in interactions with potential conflicts. While a mathematical treatment of the costs and benefits of decisions can predict the optimal strategy in simple settings, more realistic settings such as finite populations, non-vanishing mutations rates, stochastic decisions, communication between agents, and spatial interactions, require agent-based methods where each agent is modeled as an individual, carries its own genes that determine its decisions, and where the evolutionary outcome can only be ascertained by evolving the population of agents forward in time. While highlighting standard mathematical results, we compare those to agent-based methods that can go beyond the limitations of equations and simulate the complexity of heterogeneous populations and an ever-changing set of interactors. We conclude that agent-based methods can predict evolutionary outcomes where purely mathematical treatments cannot tread (for example in the weak selection-strong mutation limit), but that mathematics is crucial to validate the computational simulations.

Small groups and long memories promote cooperation

Complex social behaviors lie at the heart of many of the challenges facing evolutionary biology, sociology, economics, and beyond. For evolutionary biologists the question is often how group behaviors such as collective action, or decision making that accounts for memories of past experience, can emerge and persist in an evolving system. Evolutionary game theory provides a framework for formalizing these questions and admitting them to rigorous study. Here we develop such a framework to study the evolution of sustained collective action in multi-player public-goods games, in which players have arbitrarily long memories of prior rounds of play and can react to their experience in an arbitrary way. We construct a coordinate system for memory-m strategies in iterated n-player games that permits us to characterize all cooperative strategies that resist invasion by any mutant strategy, and stabilize cooperative behavior. We show that, especially when groups are small, longer-memory strategies make cooperation easier to evolve, by increasing the number of ways to stabilize cooperation. We also explore the co-evolution of behavior and memory. We find that even when memory has a cost, longer-memory strategies often evolve, which in turn drives the evolution of cooperation, even when the benefits for cooperation are low.

Comparing reactive and memory-one strategies of direct reciprocity

Direct reciprocity is a mechanism for the evolution of cooperation based on repeated interactions. When individuals meet repeatedly, they can use conditional strategies to enforce cooperative outcomes that would not be feasible in one-shot social dilemmas. Direct reciprocity requires that individuals keep track of their past interactions and find the right response. However, there are natural bounds on strategic complexity: Humans find it difficult to remember past interactions accurately, especially over long timespans. Given these limitations, it is natural to ask how complex strategies need to be for cooperation to evolve. Here, we study stochastic evolutionary game dynamics in finite populations to systematically compare the evolutionary performance of reactive strategies, which only respond to the co-player's previous move, and memory-one strategies, which take into account the own and the co-player's previous move. In both cases, we compare deterministic strategy and stochastic strategy spaces. For reactive strategies and small costs, we find that stochasticity benefits cooperation, because it allows for generous-tit-for-tat. For memory one strategies and small costs, we find that stochasticity does not increase the propensity for cooperation, because the deterministic rule of win-stay, lose-shift works best. For memory one strategies and large costs, however, stochasticity can augment cooperation.

Autocratic strategies for iterated games with arbitrary action spaces

The recent discovery of zero-determinant strategies for the iterated prisoner's dilemma sparked a surge of interest in the surprising fact that a player can exert unilateral control over iterated interactions. These remarkable strategies, however, are known to exist only in games in which players choose between two alternative actions such as "cooperate" and "defect." Here we introduce a broader class of autocratic strategies by extending zero-determinant strategies to iterated games with more general action spaces. We use the continuous donation game as an example, which represents an instance of the prisoner's dilemma that intuitively extends to a continuous range of cooperation levels. Surprisingly, despite the fact that the opponent has infinitely many donation levels from which to choose, a player can devise an autocratic strategy to enforce a linear relationship between his or her payoff and that of the opponent even when restricting his or her actions to merely two discrete levels of cooperation. In particular, a player can use such a strategy to extort an unfair share of the payoffs from the opponent. Therefore, although the action space of the continuous donation game dwarfs that of the classic prisoner's dilemma, players can still devise relatively simple autocratic and, in particular, extortionate strategies.

Humans choose representatives who enforce cooperation in social dilemmas through extortion

Social dilemmas force players to balance between personal and collective gain. In many dilemmas, such as elected governments negotiating climate-change mitigation measures, the decisions are made not by individual players but by their representatives. However, the behaviour of representatives in social dilemmas has not been investigated experimentally. Here inspired by the negotiations for greenhouse-gas emissions reductions, we experimentally study a collective-risk social dilemma that involves representatives deciding on behalf of their fellow group members. Representatives can be re-elected or voted out after each consecutive collective-risk game. Selfish players are preferentially elected and are hence found most frequently in the 'representatives' treatment. Across all treatments, we identify the selfish players as extortioners. As predicted by our mathematical model, their steadfast strategies enforce cooperation from fair players who finally compensate almost completely the deficit caused by the extortionate co-players. Everybody gains, but the extortionate representatives and their groups gain the most.

Evolution of Conformity in Social Dilemmas

People often deviate from their individual Nash equilibrium strategy in game experiments based on the prisoner’s dilemma (PD) game and the public goods game (PGG), whereas conditional cooperation, or conformity, is supported by the data from these experiments. In a complicated environment with no obvious “dominant” strategy, conformists who choose the average strategy of the other players in their group could be able to avoid risk by guaranteeing their income will be close to the group average. In this paper, we study the repeated PD game and the repeated m-person PGG, where individuals’ strategies are restricted to the set of conforming strategies. We define a conforming strategy by two parameters, initial action in the game and the influence of the other players’ choices in the previous round. We are particularly interested in the tit-for-tat (TFT) strategy, which is the well-known conforming strategy in theoretical and empirical studies. In both the PD game and the PGG, TFT can prevent the invasion of non-cooperative strategy if the expected number of rounds exceeds a critical value. The stability analysis of adaptive dynamics shows that conformity in general promotes the evolution of cooperation, and that a regime of cooperation can be established in an AllD population through TFT-like strategies. These results provide insight into the emergence of cooperation in social dilemma games.

Zero-Determinant Strategies in Iterated Public Goods Game

Recently, Press and Dyson have proposed a new class of probabilistic and conditional strategies for the two-player iterated Prisoner’s Dilemma, so-called zero-determinant strategies. A player adopting zero-determinant strategies is able to pin the expected payoff of the opponents or to enforce a linear relationship between his own payoff and the opponents’ payoff, in a unilateral way. This paper considers zero-determinant strategies in the iterated public goods game, a representative multi-player game where in each round each player will choose whether or not to put his tokens into a public pot, and the tokens in this pot are multiplied by a factor larger than one and then evenly divided among all players. The analytical and numerical results exhibit a similar yet different scenario to the case of two-player games: (i) with small number of players or a small multiplication factor, a player is able to unilaterally pin the expected total payoff of all other players; (ii) a player is able to set the ratio between his payoff and the total payoff of all other players, but this ratio is limited by an upper bound if the multiplication factor exceeds a threshold that depends on the number of players.