Memory-n strategies of direct reciprocity

Evolution of reciprocity with limited payoff memory

Direct reciprocity is a mechanism for the evolution of cooperation in repeated social interactions. According to the literature, individuals naturally learn to adopt conditionally cooperative strategies if they have multiple encounters with their partner. Corresponding models have greatly facilitated our understanding of cooperation, yet they often make strong assumptions on how individuals remember and process payoff information. For example, when strategies are updated through social learning, it is commonly assumed that individuals compare their average payoffs. This would require them to compute (or remember) their payoffs against everyone else in the population. To understand how more realistic constraints influence direct reciprocity, we consider the evolution of conditional behaviours when individuals learn based on more recent experiences. Even in the most extreme case that they only take into account their very last interaction, we find that cooperation can still evolve. However, such individuals adopt less generous strategies, and they cooperate less often than in the classical setup with average payoffs. Interestingly, once individuals remember the payoffs of two or three recent interactions, cooperation rates quickly approach the classical limit. These findings contribute to a literature that explores which kind of cognitive capabilities are required for reciprocal cooperation. While our results suggest that some rudimentary form of payoff memory is necessary, it suffices to remember a few interactions.

Resolving selfish and spiteful interdependent conflict

Interdependence occurs when individuals have a stake in the success or failure of others, such that the outcomes experienced by one individual also generate costs or benefits for others. Discussion on this topic has typically focused on positive interdependence (where gains for one individual result in gains for another) and on the consequences for cooperation. However, interdependence can also be negative (where gains for one individual result in losses for another), which can spark conflict. In this article, we explain when negative interdependence is likely to arise and, crucially, the role played by (mis)perception in shaping an individual's understanding of their interdependent relationships. We argue that, owing to the difficulty in accurately perceiving interdependence with others, individuals might often be mistaken about the stake they hold in each other's outcomes, which can spark needless, resolvable forms of conflict. We then discuss when and how reducing misperceptions can help to resolve such conflicts. We argue that a key mechanism for resolving interdependent conflict, along with better sources of exogenous information, is to reduce reliance on heuristics such as stereotypes when assessing the nature of our interdependent relationships.

Tax thresholds yield multiple optimal cooperation levels in the spatial public goods game

Income redistribution, which involves transferring income from certain individuals to others, plays a crucial role in human societies. Previous research has indicated that tax-based redistribution can promote cooperation by enhancing incentives for cooperators. In such a tax system, all individuals, irrespective of their income levels, contribute to the tax system, and the tax revenue is subsequently redistributed to everyone. In this study, we relax this assumption by introducing a tax threshold, signifying that only individuals with incomes exceeding the threshold will be subject to taxation. In particular, we employ the spatial public goods game to investigate the influence of tax rates-the percentage of income allocated to tax-and tax thresholds, which determine the income level at which individuals become taxable, on the evolution of cooperation. Our extensive numerical simulations disclose that tax thresholds produce complex outcomes for the evolution of cooperation, depending on tax rates. Notably, at low tax rates (i.e., below 0.41), as the tax threshold increases, discontinuous phase transitions in cooperation performance suggest the presence of multiple intervals of effective tax thresholds that promote peak cooperation levels. Nevertheless, irrespective of the chosen tax rate, once the tax threshold surpasses a critical threshold, the redistribution mechanism fails, causing the collapse of cooperation. Evolutionary snapshots show that self-organized redistribution forms an intermediary layer on the peripheries of cooperative clusters, effectively shielding cooperators from potential defectors. Quantitative analyses shed light on how self-organized redistribution narrows the income gap between cooperators and defectors through precise identification of tax-exempt entities, thereby amplifying the cooperative advantage. Collectively, these findings enhance our comprehension of how income redistribution influences cooperation, highlighting the pivotal role of tax thresholds.

A geometric process of evolutionary game dynamics

Many evolutionary processes occur in phenotype spaces which are continuous. It is therefore of interest to explore how selection operates in continuous spaces. One approach is adaptive dynamics, which assumes that mutants are local. Here we study a different process which also allows non-local mutants. We assume that a resident population is challenged by an invader who uses a strategy chosen from a random distribution on the space of all strategies. We study the repeated donation game of direct reciprocity. We consider reactive strategies given by two probabilities, denoting respectively the probability to cooperate after the co-player has cooperated or defected. The strategy space is the unit square. We derive analytic formulae for the stationary distribution of evolutionary dynamics and for the average cooperation rate as function of the cost-to-benefit ratio. For positive reactive strategies, we prove that cooperation is more abundant than defection if the area of the cooperative region is greater than 1/2 which is equivalent to benefit, b, divided by cost, c, exceeding [Formula: see text]. We introduce the concept of strategies that are stable with probability one. We also study an extended process and discuss other games.

The effect of environmental information on evolution of cooperation in stochastic games

Many human interactions feature the characteristics of social dilemmas where individual actions have consequences for the group and the environment. The feedback between behavior and environment can be studied with the framework of stochastic games. In stochastic games, the state of the environment can change, depending on the choices made by group members. Past work suggests that such feedback can reinforce cooperative behaviors. In particular, cooperation can evolve in stochastic games even if it is infeasible in each separate repeated game. In stochastic games, participants have an interest in conditioning their strategies on the state of the environment. Yet in many applications, precise information about the state could be scarce. Here, we study how the availability of information (or lack thereof) shapes evolution of cooperation. Already for simple examples of two state games we find surprising effects. In some cases, cooperation is only possible if there is precise information about the state of the environment. In other cases, cooperation is most abundant when there is no information about the state of the environment. We systematically analyze all stochastic games of a given complexity class, to determine when receiving information about the environment is better, neutral, or worse for evolution of cooperation.

Grouping promotes both partnership and rivalry with long memory in direct reciprocity

Biological and social scientists have long been interested in understanding how to reconcile individual and collective interests in the iterated Prisoner's Dilemma. Many effective strategies have been proposed, and they are often categorized into one of two classes, 'partners' and 'rivals.' More recently, another class, 'friendly rivals,' has been identified in longer-memory strategy spaces. Friendly rivals qualify as both partners and rivals: They fully cooperate with themselves, like partners, but never allow their co-players to earn higher payoffs, like rivals. Although they have appealing theoretical properties, it is unclear whether they would emerge in an evolving population because most previous works focus on the memory-one strategy space, where no friendly rival strategy exists. To investigate this issue, we have conducted evolutionary simulations in well-mixed and group-structured populations and compared the evolutionary dynamics between memory-one and longer-memory strategy spaces. In a well-mixed population, the memory length does not make a major difference, and the key factors are the population size and the benefit of cooperation. Friendly rivals play a minor role because being a partner or a rival is often good enough in a given environment. It is in a group-structured population that memory length makes a stark difference: When longer-memory strategies are available, friendly rivals become dominant, and the cooperation level nearly reaches a maximum, even when the benefit of cooperation is so low that cooperation would not be achieved in a well-mixed population. This result highlights the important interaction between group structure and memory lengths that drive the evolution of cooperation.

Mutation enhances cooperation in direct reciprocity

Direct reciprocity is a powerful mechanism for the evolution of cooperation based on repeated interactions between the same individuals. But high levels of cooperation evolve only if the benefit-to-cost ratio exceeds a certain threshold that depends on memory length. For the best-explored case of one-round memory, that threshold is two. Here, we report that intermediate mutation rates lead to high levels of cooperation, even if the benefit-to-cost ratio is only marginally above one, and even if individuals only use a minimum of past information. This surprising observation is caused by two effects. First, mutation generates diversity which undermines the evolutionary stability of defectors. Second, mutation leads to diverse communities of cooperators that are more resilient than homogeneous ones. This finding is relevant because many real-world opportunities for cooperation have small benefit-to-cost ratios, which are between one and two, and we describe how direct reciprocity can attain cooperation in such settings. Our result can be interpreted as showing that diversity, rather than uniformity, promotes evolution of cooperation.

Evolution of direct reciprocity in group-structured populations

People tend to have their social interactions with members of their own community. Such group-structured interactions can have a profound impact on the behaviors that evolve. Group structure affects the way people cooperate, and how they reciprocate each other's cooperative actions. Past work has shown that population structure and reciprocity can both promote the evolution of cooperation. Yet the impact of these mechanisms has been typically studied in isolation. In this work, we study how the two mechanisms interact. Using a game-theoretic model, we explore how people engage in reciprocal cooperation in group-structured populations, compared to well-mixed populations of equal size. In this model, the population is subdivided into groups. Individuals engage in pairwise interactions within groups while they also have chances to imitate strategies outside the groups. To derive analytical results, we focus on two scenarios. In the first scenario, we assume a complete separation of time scales. Mutations are rare compared to between-group comparisons, which themselves are rare compared to within-group comparisons. In the second scenario, there is a partial separation of time scales, where mutations and between-group comparisons occur at a comparable rate. In both scenarios, we find that the effect of population structure depends on the benefit of cooperation. When this benefit is small, group-structured populations are more cooperative. But when the benefit is large, well-mixed populations result in more cooperation. Overall, our results reveal how group structure can sometimes enhance and sometimes suppress the evolution of cooperation.

Evolution of cooperation through cumulative reciprocity

Reciprocity is a simple principle for cooperation that explains many of the patterns of how humans seek and receive help from each other. To capture reciprocity, traditional models often assume that individuals use simple strategies with restricted memory. These memory-1 strategies are mathematically convenient, but they miss important aspects of human reciprocity, where defections can have lasting effects. Here we instead propose a strategy of cumulative reciprocity. Cumulative reciprocators count the imbalance of cooperation across their previous interactions with their opponent. They cooperate as long as this imbalance is sufficiently small. Using analytical and computational methods, we show that this strategy can sustain cooperation in the presence of errors, that it enforces fair outcomes and that it evolves in hostile environments. Using an economic experiment, we confirm that cumulative reciprocity is more predictive of human behaviour than several classical strategies. The basic principle of cumulative reciprocity is versatile and can be extended to a range of social dilemmas.

Evolutionary instability of selfish learning in repeated games

Across many domains of interaction, both natural and artificial, individuals use past experience to shape future behaviors. The results of such learning processes depend on what individuals wish to maximize. A natural objective is one's own success. However, when two such "selfish" learners interact with each other, the outcome can be detrimental to both, especially when there are conflicts of interest. Here, we explore how a learner can align incentives with a selfish opponent. Moreover, we consider the dynamics that arise when learning rules themselves are subject to evolutionary pressure. By combining extensive simulations and analytical techniques, we demonstrate that selfish learning is unstable in most classical two-player repeated games. If evolution operates on the level of long-run payoffs, selection instead favors learning rules that incorporate social (other-regarding) preferences. To further corroborate these results, we analyze data from a repeated prisoner's dilemma experiment. We find that selfish learning is insufficient to explain human behavior when there is a trade-off between payoff maximization and fairness.

Evolutionary dynamics of zero-determinant strategies in repeated multiplayer games

Several studies have confirmed the existence of zero-determinant (ZD) strategies in repeated social dilemmas since Press and Dyson's ingenious discovery of ZD strategies in iterated prisoner's dilemmas. However, less research studies evolutionary performance of multiplayer ZD strategies, especially from a theoretical perspective. Here, we use a state-clustering method to theoretically analyze evolutionary dynamics of two representative ZD strategies: generous ZD strategies and extortionate ZD strategies. We consider two new settings for multiplayer ZD strategies: competitions with all ZD strategies and competitions with all memory-one strategies, apart from the competitions between these strategies and some classical ones. Moreover, we investigate the influence of the level of generosity and extortion on evolutionary dynamics of generous and extortionate ZD strategies, which was commonly ignored in previous studies. Theoretical results show that players with limited generosity are at an advantageous place and extortioners extorting more severely hold their ground more readily. Our results may provide new insights into better understanding evolutionary dynamics of ZD strategies in repeated multiplayer games.

Direct reciprocity between individuals that use different strategy spaces

In repeated interactions, players can use strategies that respond to the outcome of previous rounds. Much of the existing literature on direct reciprocity assumes that all competing individuals use the same strategy space. Here, we study both learning and evolutionary dynamics of players that differ in the strategy space they explore. We focus on the infinitely repeated donation game and compare three natural strategy spaces: memory-1 strategies, which consider the last moves of both players, reactive strategies, which respond to the last move of the co-player, and unconditional strategies. These three strategy spaces differ in the memory capacity that is needed. We compute the long term average payoff that is achieved in a pairwise learning process. We find that smaller strategy spaces can dominate larger ones. For weak selection, unconditional players dominate both reactive and memory-1 players. For intermediate selection, reactive players dominate memory-1 players. Only for strong selection and low cost-to-benefit ratio, memory-1 players dominate the others. We observe that the supergame between strategy spaces can be a social dilemma: maximum payoff is achieved if both players explore a larger strategy space, but smaller strategy spaces dominate.

Direct reciprocity can lead to cooperation between individuals who meet in repeated encounters. The shadow of the future casts an incentive to cooperate. If I cooperate today, you may cooperate tomorrow. But if I defect today, you may defect tomorrow. In most studies of direct reciprocity it is assumed that both players explore the same space of possible strategies. In contrast, here we study interactions between players that use different strategy spaces and therefore utilize different memory capacities. Surprisingly, we find that more complex strategy spaces often lose out against simpler ones. The social optimum, however, is achieved if all players use the more complex space. Therefore, the game between strategy spaces becomes a higher order social dilemma.

Payoff landscapes and the robustness of selfish optimization in iterated games

In iterated games, a player can unilaterally exert influence over the outcome through a careful choice of strategy. A powerful class of such "payoff control" strategies was discovered by Press and Dyson (2012). Their so-called "zero-determinant" (ZD) strategies allow a player to unilaterally enforce a linear relationship between both players' payoffs. It was subsequently shown by Chen and Zinger (2014) that when the slope of this linear relationship is positive, ZD strategies are robustly effective against a selfishly optimizing co-player, in that all adapting paths of the selfish player lead to the maximal payoffs for both players (at least when there are certain restrictions on the game parameters). In this paper, we investigate the efficacy of selfish learning against a fixed player in more general settings, for both ZD and non-ZD strategies. We first prove that in any symmetric 2[Formula: see text]2 game, the selfish player's final strategy must be of a certain form and cannot be fully stochastic. We then show that there are prisoner's dilemma interactions for which selfish optimization does not always lead to maximal payoffs against fixed ZD strategies with positive slope. We give examples of selfish adapting paths that lead to locally but not globally optimal payoffs, undermining the robustness of payoff control strategies. For non-ZD strategies, these pathologies arise regardless of the original restrictions on the game parameters. Our results illuminate the difficulty of implementing robust payoff control and selfish optimization, even in the simplest context of playing against a fixed strategy.

Evolution of cooperation with joint liability

'Personal responsibility', one of the basic principles of social governance, requires one to be accountable for what one does. However, personal responsibility is far from the only norm ruling human interactions, especially in social and economic activities. In many collective communities such as among enterprise colleagues and family members, one's personal interests are often bound to others'-once one member breaks the rule, a group of people have to bear the punishment or sanction. Such a mechanism is termed 'joint liability'. Although many real-world cases have evidenced that joint liability can help to maintain collective collaboration, a deep and systematic theoretical analysis on how and when it promotes cooperation remains lacking. Here, we use evolutionary game theory to model an interacting system with joint liability, where one's losing credit could deteriorate the reputation of the whole group. We provide the analytical condition to predict when cooperation evolves and analytically prove that in the presence of punishment, being jointly liable greatly promotes cooperation. Our work stresses that joint liability is of great significance in promoting current economic prosperity.

Cooperation in alternating interactions with memory constraints

In repeated social interactions, individuals often employ reciprocal strategies to maintain cooperation. To explore the emergence of reciprocity, many theoretical models assume synchronized decision making. In each round, individuals decide simultaneously whether to cooperate or not. Yet many manifestations of reciprocity in nature are asynchronous. Individuals provide help at one time and receive help at another. Here, we explore such alternating games in which players take turns. We mathematically characterize all Nash equilibria among memory-one strategies. Moreover, we use evolutionary simulations to explore various model extensions, exploring the effect of discounted games, irregular alternation patterns, and higher memory. In all cases, we observe that mutual cooperation still evolves for a wide range of parameter values. However, compared to simultaneous games, alternating games require different strategies to maintain cooperation in noisy environments. Moreover, none of the respective strategies are evolutionarily stable.

In many instances of reciprocity, individuals cooperate in turns. Here, the authors analyze the equilibria and the dynamics of such alternating games, and in particular describe all strategies with one-round memory that maintain cooperation.

The evolution of moral rules in a model of indirect reciprocity with private assessment

Moral rules allow humans to cooperate by indirect reciprocity. Yet, it is not clear which moral rules best implement indirect reciprocity and are favoured by natural selection. Previous studies either considered only public assessment, where individuals are deemed good or bad by all others, or compared a subset of possible strategies. Here we fill this gap by identifying which rules are evolutionary stable strategies (ESS) among all possible moral rules while considering private assessment. We develop an analytical model describing the frequency of long-term cooperation, determining when a strategy can be invaded by another. We show that there are numerous ESSs in absence of errors, which however cease to exist when errors are present. We identify the underlying properties of cooperative ESSs. Overall, this paper provides a first exhaustive evolutionary invasion analysis of moral rules considering private assessment. Moreover, this model is extendable to incorporate higher-order rules and other processes.

A unified framework of direct and indirect reciprocity

Direct and indirect reciprocity are key mechanisms for the evolution of cooperation. Direct reciprocity means that individuals use their own experience to decide whether to cooperate with another person. Indirect reciprocity means that they also consider the experiences of others. Although these two mechanisms are intertwined, they are typically studied in isolation. Here, we introduce a mathematical framework that allows us to explore both kinds of reciprocity simultaneously. We show that the well-known 'generous tit-for-tat' strategy of direct reciprocity has a natural analogue in indirect reciprocity, which we call 'generous scoring'. Using an equilibrium analysis, we characterize under which conditions either of the two strategies can maintain cooperation. With simulations, we additionally explore which kind of reciprocity evolves when members of a population engage in social learning to adapt to their environment. Our results draw unexpected connections between direct and indirect reciprocity while highlighting important differences regarding their evolvability.

Memory as a cognitive requirement for reciprocal cooperation

Memory has evolved to guide our decisions in the present and to prepare us for future interactions with the environment. Within the social domain, memory can help to decide with whom to cooperate. This provides a unique opportunity to study memory from a functional perspective. Although several lines of research have demonstrated that many forms of reciprocal cooperation require memory, most of the research does not support the assumption of a highly specialized cheater-detection module that specifically serves to promote the detection of uncooperative interaction partners. Instead, the literature supports the flexible recruitment of domain-general guessing and memory mechanisms that serve to continuously predict the future behavior of others based on situational and person-specific factors and use violations of these expectations to update the predictive models of who can be trusted to cooperate in reciprocal interactions.

Limited memory optimizes cooperation in social dilemma experiments

Cooperation is one of the key collective behaviours of human society. Despite discoveries of several social mechanisms underpinning cooperation, relatively little is known about how our neural functions affect cooperative behaviours. Here, we study the effect of a main neural function, working-memory capacity, on cooperation in repeated Prisoner's Dilemma experiments. Our experimental paradigm overcomes the obstacles in measuring and changing subjects' working-memory capacity. We find that the optimal cooperation level occurs when subjects remember two previous rounds of information, and cooperation increases abruptly from no memory capacity to minimal memory capacity. The results can be explained by memory-based conditional cooperation of subjects. We propose evolutionary models based on replicator dynamics and Markov processes, respectively, which are in good agreement with experimental results of different memory capacities. Our experimental findings differ from previous hypotheses and predictions of existent models and theories, and suggest a neural basis and evolutionary roots of cooperation beyond cultural influences.

Evolution of state-dependent strategies in stochastic games

In a population of interacting individuals, the environment for interactions often changes due to individuals' behaviors, which in turn drive the evolution of individuals' behaviors. The interplay between the environment and individuals' behaviors has been demonstrated to remarkably influence the evolutionary outcomes. In reality, in highly cognitive species such as social primates and human beings, individuals are often capable of perceiving the environment change and then differentiate their strategies across different environment states. We propose a model of environmental feedback with state-dependent strategies: individuals have perceptions of distinct environment states and therefore take distinct sub-strategies under each of them; based on the sub-strategy, individuals then decide their behaviors; their behaviors subsequently modify the environment state. We use the theory of stochastic games and evolutionary dynamics to analyze this idea. We find that when environment changes slower than behaviors, state-dependent strategies (i.e. taking different sub-strategies under different environment states) can outperform state-independent strategies (i.e. taking an identical sub-strategy under all environment states), such as Win-Stay, Lose-Shift, the most leading strategy among state-independent strategies. The intuition is that delayed environmental feedback provides chances for individuals with state-dependent strategies to exploit those with state-independent strategies. Our results hold (1) in both well-mixed and structured populations; (2) when the environment switches between two or more states. Furthermore, the environment changing rate decides if state-dependent strategies benefit global cooperation. The evolution sees the rise of the cooperation level for fast environment switching and the decrease otherwise. Our work stresses that individuals' perceptions of different environment states are beneficial to their survival and social prosperity in a changing world.

Memory-two zero-determinant strategies in repeated games

Repeated games have provided an explanation of how mutual cooperation can be achieved even if defection is more favourable in a one-shot game in the Prisoner's Dilemma situation. Recently found zero-determinant (ZD) strategies have substantially been investigated in evolutionary game theory. The original memory-one ZD strategies unilaterally enforce linear relationships between average pay-offs of players. Here, we extend the concept of ZD strategies to memory-two strategies in repeated games. Memory-two ZD strategies unilaterally enforce linear relationships between correlation functions of pay-offs and pay-offs of the previous round. Examples of memory-two ZD strategy in the repeated Prisoner's Dilemma game are provided, some of which generalize the tit-for-tat strategy to a memory-two case. Extension of ZD strategies to memory-n case with n ≥ ~2 is also straightforward.

Friendly-rivalry solution to the iterated n-person public-goods game

Repeated interaction promotes cooperation among rational individuals under the shadow of future, but it is hard to maintain cooperation when a large number of error-prone individuals are involved. One way to construct a cooperative Nash equilibrium is to find a ‘friendly-rivalry’ strategy, which aims at full cooperation but never allows the co-players to be better off. Recently it has been shown that for the iterated Prisoner’s Dilemma in the presence of error, a friendly rival can be designed with the following five rules: Cooperate if everyone did, accept punishment for your own mistake, punish defection, recover cooperation if you find a chance, and defect in all the other circumstances. In this work, we construct such a friendly-rivalry strategy for the iterated n-person public-goods game by generalizing those five rules. The resulting strategy makes a decision with referring to the previous m = 2n − 1 rounds. A friendly-rivalry strategy for n = 2 inherently has evolutionary robustness in the sense that no mutant strategy has higher fixation probability in this population than that of a neutral mutant. Our evolutionary simulation indeed shows excellent performance of the proposed strategy in a broad range of environmental conditions when n = 2 and 3.

How to maintain cooperation among a number of self-interested individuals is a difficult problem, especially if they can sometimes commit error. In this work, we propose a strategy for the iterated n-person public-goods game based on the following five rules: Cooperate if everyone did, accept punishment for your own mistake, punish others’ defection, recover cooperation if you find a chance, and defect in all the other circumstances. These rules are not far from actual human behavior, and the resulting strategy guarantees three advantages: First, if everyone uses it, full cooperation is recovered even if error occurs with small probability. Second, the player of this strategy always never obtains a lower long-term payoff than any of the co-players. Third, if the co-players are unconditional cooperators, it obtains a strictly higher long-term payoff than theirs. Therefore, if everyone uses this strategy, no one has a reason to change it. Furthermore, our simulation shows that this strategy will become highly abundant over long time scales due to its robustness against the invasion of other strategies. In this sense, the repeated social dilemma is solved for an arbitrary number of players.

Signalling boosts the evolution of cooperation in repeated group interactions

Many biological and social systems show significant levels of collective action. Several cooperation mechanisms have been proposed, yet they have been mostly studied independently. Among these, direct reciprocity supports cooperation on the basis of repeated interactions among individuals. Signals and quorum dynamics may also drive cooperation. Here, we resort to an evolutionary game-theoretical model to jointly analyse these two mechanisms and study the conditions in which evolution selects for direct reciprocity, signalling, or their combination. We show that signalling alone leads to higher levels of cooperation than when combined with reciprocity, while offering additional robustness against errors. Specifically, successful strategies in the realm of direct reciprocity are often not selected in the presence of signalling, and memory of past interactions is only exploited opportunistically in the case of earlier coordination failure. Differently, signalling always evolves, even when costly. In the light of these results, it may be easier to understand why direct reciprocity has been observed only in a limited number of cases among non-humans, whereas signalling is widespread at all levels of complexity.

Five rules for friendly rivalry in direct reciprocity

Direct reciprocity is one of the key mechanisms accounting for cooperation in our social life. According to recent understanding, most of classical strategies for direct reciprocity fall into one of two classes, ‘partners’ or ‘rivals’. A ‘partner’ is a generous strategy achieving mutual cooperation, and a ‘rival’ never lets the co-player become better off. They have different working conditions: For example, partners show good performance in a large population, whereas rivals do in head-to-head matches. By means of exhaustive enumeration, we demonstrate the existence of strategies that act as both partners and rivals. Among them, we focus on a human-interpretable strategy, named ‘CAPRI’ after its five characteristic ingredients, i.e., cooperate, accept, punish, recover, and defect otherwise. Our evolutionary simulation shows excellent performance of CAPRI in a broad range of environmental conditions.

Zero-determinant strategies under observation errors in repeated games

Zero-determinant (ZD) strategies are a novel class of strategies in the repeated prisoner's dilemma (RPD) game discovered by Press and Dyson. This strategy set enforces a linear payoff relationship between a focal player and the opponent regardless of the opponent's strategy. In the RPD game, games with discounting and observation errors represent an important generalization, because they are better able to capture real life interactions which are often noisy. However, they have not been considered in the original discovery of ZD strategies. In some preceding studies, each of them has been considered independently. Here, we analytically study the strategies that enforce linear payoff relationships in the RPD game considering both a discount factor and observation errors. As a result, we first reveal that the payoffs of two players can be represented by the form of determinants as shown by Press and Dyson even with the two factors. Then, we search for all possible strategies that enforce linear payoff relationships and find that both ZD strategies and unconditional strategies are the only strategy sets to satisfy the condition. We also show that neither Extortion nor Generous strategies, which are subsets of ZD strategies, exist when there are errors. Finally, we numerically derive the threshold values above which the subsets of ZD strategies exist. These results contribute to a deep understanding of ZD strategies in society.

Automata representation of successful strategies for social dilemmas

In a social dilemma, cooperation is collectively optimal, yet individually each group member prefers to defect. A class of successful strategies of direct reciprocity were recently found for the iterated prisoner’s dilemma and for the iterated three-person public-goods game: By a successful strategy, we mean that it constitutes a cooperative Nash equilibrium under implementation error, with assuring that the long-term payoff never becomes less than the co-players’ regardless of their strategies, when the error rate is small. Although we have a list of actions prescribed by each successful strategy, the rationale behind them has not been fully understood for the iterated public-goods game because the list has hundreds of entries to deal with every relevant history of previous interactions. In this paper, we propose a method to convert such history-based representation into an automaton with a minimal number of states. Our main finding is that a successful strategy for the iterated three-person public-goods game can be represented as a 10-state automaton by this method. In this automaton, each state can be interpreted as the player’s internal judgement of the situation, such as trustworthiness of the co-players and the need to redeem oneself after defection. This result thus suggests a comprehensible way to choose an appropriate action at each step towards cooperation based on a situational judgement, which is mapped from the history of interactions.

Evolving cooperation in multichannel games

Humans routinely engage in many distinct interactions in parallel. Team members collaborate on several concurrent projects, and even whole nations interact with each other across a variety of issues, including trade, climate change and security. Yet the existing theory of direct reciprocity studies isolated repeated games. Such models cannot account for strategic attempts to use the vested interests in one game as a leverage to enforce cooperation in another. Here we introduce a general framework of multichannel games. Individuals interact with each other over multiple channels; each channel is a repeated game. Strategic choices in one channel can affect decisions in another. With analytical equilibrium calculations for the donation game and evolutionary simulations for several other games we show that such linkage facilitates cooperation. Our results suggest that previous studies tend to underestimate the human potential for reciprocity. When several interactions occur in parallel, people often learn to coordinate their behavior across games to maximize cooperation in each of them.

Evolutionary dynamics drives role specialization in a community of players

The progression of game theory from classical to evolutionary and spatial games provided a powerful means to study cooperation, and enabled a better understanding of general cooperation-promoting mechanisms. However, current standard models assume that at any given point players must choose either cooperation or defection, meaning that regardless of the spatial structure in which they exist, they cannot differentiate between their neighbours and adjust their behaviour accordingly. This is at odds with interactions among organisms in nature who are well capable of behaving differently towards different members of their communities. We account for this natural fact by introducing a new type of player-dubbed link players-who can adjust their behaviour to each individual neighbour. This is in contrast to more common node players whose behaviour affects all neighbours in the same way. We proceed to study cooperation in pure and mixed populations, showing that cooperation peaks at moderately low densities of link players. In such conditions, players naturally specialize in different roles. Node players tend to be either cooperators or defectors, while link players form social insulation between cooperative and defecting clusters by acting both as cooperators and defectors. Such fairly complex processes emerging from a simple model reflect some of the complexities observed in experimental studies on social behaviour in microbes and pave a way for the development of richer game models.

Linear algebraic structure of zero-determinant strategies in repeated games

Zero-determinant (ZD) strategies, a recently found novel class of strategies in repeated games, has attracted much attention in evolutionary game theory. A ZD strategy unilaterally enforces a linear relation between average payoffs of players. Although existence and evolutional stability of ZD strategies have been studied in simple games, their mathematical properties have not been well-known yet. For example, what happens when more than one players employ ZD strategies have not been clarified. In this paper, we provide a general framework for investigating situations where more than one players employ ZD strategies in terms of linear algebra. First, we theoretically prove that a set of linear relations of average payoffs enforced by ZD strategies always has solutions, which implies that incompatible linear relations are impossible. Second, we prove that linear payoff relations are independent of each other under some conditions. These results hold for general games with public monitoring including perfect-monitoring games. Furthermore, we provide a simple example of a two-player game in which one player can simultaneously enforce two linear relations, that is, simultaneously control her and her opponent's average payoffs. All of these results elucidate general mathematical properties of ZD strategies.

Emergence and suppression of cooperation by action visibility in transparent games

Real-world agents, humans as well as animals, observe each other during interactions and choose their own actions taking the partners' ongoing behaviour into account. Yet, classical game theory assumes that players act either strictly sequentially or strictly simultaneously without knowing each other's current choices. To account for action visibility and provide a more realistic model of interactions under time constraints, we introduce a new game-theoretic setting called transparent games, where each player has a certain probability of observing the partner's choice before deciding on its own action. By means of evolutionary simulations, we demonstrate that even a small probability of seeing the partner's choice before one's own decision substantially changes the evolutionary successful strategies. Action visibility enhances cooperation in an iterated coordination game, but reduces cooperation in a more competitive iterated Prisoner's Dilemma. In both games, "Win-stay, lose-shift" and "Tit-for-tat" strategies are predominant for moderate transparency, while a "Leader-Follower" strategy emerges for high transparency. Our results have implications for studies of human and animal social behaviour, especially for the analysis of dyadic and group interactions.

Individual perception dynamics in drunk games

We study the effects of individual perceptions of payoffs in two-player games. In particular we consider the setting in which individuals' perceptions of the game are influenced by their previous experiences and outcomes. Accordingly, we introduce a framework based on evolutionary games where individuals have the capacity to perceive their interactions in different ways. Starting from the narrative of social behaviors in a pub as an illustration, we first study the combination of the Prisoner's Dilemma and Harmony Game as two alternative perceptions of the same situation. Considering a selection of game pairs, our results show that the interplay between perception dynamics and game payoffs gives rise to nonlinear phenomena unexpected in each of the games separately, such as catastrophic phase transitions in the cooperation basin of attraction, Hopf bifurcations and cycles of cooperation and defection. Combining analytical techniques with multiagent simulations, we also show how introducing individual perceptions can cause nontrivial dynamical behaviors to emerge, which cannot be obtained by analyzing the system at a macroscopic level. Specifically, initial perception heterogeneities at the microscopic level can yield a polarization effect that is unpredictable at the macroscopic level. This framework opens the door to the exploration of new ways of understanding the link between the emergence of cooperation and individual preferences and perceptions, with potential applications beyond social interactions.

Reactive learning strategies for iterated games

In an iterated game between two players, there is much interest in characterizing the set of feasible pay-offs for both players when one player uses a fixed strategy and the other player is free to switch. Such characterizations have led to extortionists, equalizers, partners and rivals. Most of those studies use memory-one strategies, which specify the probabilities to take actions depending on the outcome of the previous round. Here, we consider 'reactive learning strategies', which gradually modify their propensity to take certain actions based on past actions of the opponent. Every linear reactive learning strategy, p *, corresponds to a memory one-strategy, p , and vice versa. We prove that for evaluating the region of feasible pay-offs against a memory-one strategy, C ( p ) , we need to check its performance against at most 11 other strategies. Thus, C ( p ) is the convex hull inR 2 of at most 11 points. Furthermore, if p is a memory-one strategy, with feasible pay-off region C ( p ) , and p * is the corresponding reactive learning strategy, with feasible pay-off region C ( p ∗ ) , then C ( p ∗ ) is a subset of C ( p ) . Reactive learning strategies are therefore powerful tools in restricting the outcomes of iterated games.

Escalation of Memory Length in Finite Populations

The escalation of complexity is a commonly cited benefit of coevolutionary systems, but computational simulations generally fail to demonstrate this capacity to a satisfactory degree. We draw on a macroevolutionary theory of escalation to develop a set of criteria for coevolutionary systems to exhibit escalation of strategic complexity. By expanding on a previously developed model of the evolution of memory length for cooperative strategies by Kristian Lindgren, we resolve previously observed limitations on the escalation of memory length by extending operators of evolutionary variation. We present long-term coevolutionary simulations showing that larger population sizes tend to support greater escalation of complexity than smaller ones do. Additionally, we investigate the sensitivity of escalation during transitions of complexity. The Lindgren model has often been used to argue that the escalation of competitive coevolution has intrinsic limitations. Our simulations show that coevolutionary arms races can continue to escalate in computational simulations given sufficient population sizes.

Investment preference promotes cooperation in spatial public goods game

It is usually assumed that each cooperator contributes equally to different public pools in spatial public goods game. However, it is more reasonable to invest differently according to individual investment preference. In this paper, an extended public goods game, in which cooperators contribute to the groups according to the investment preference, is developed. The investment preference of a cooperator is characterized by the fraction of the cooperator from his/her own memory about a group and the intensity of investment preference is represented by a tunable parameter α. The well-mixed population and the structured population are analyzed under this mechanism. It is shown that the investment preference can give rise to coordination. Moreover, the extensive numerical simulation results show that with the increasing of investment preference density or memory length, the proportion of cooperation can increase monotonously. This is because the investment preference could help cooperators resist the invasion from defectors. Compared with the basic version, the new mechanism is able to promote cooperation effectively. Our research may provide a valuable insight for further exploring the nature of cooperation in the real world.

Cost-effective external interference for promoting the evolution of cooperation

The problem of promoting the evolution of cooperative behaviour within populations of self-regarding individuals has been intensively investigated across diverse fields of behavioural, social and computational sciences. In most studies, cooperation is assumed to emerge from the combined actions of participating individuals within the populations, without taking into account the possibility of external interference and how it can be performed in a cost-efficient way. Here, we bridge this gap by studying a cost-efficient interference model based on evolutionary game theory, where an exogenous decision-maker aims to ensure high levels of cooperation from a population of individuals playing the one-shot Prisoner’s Dilemma, at a minimal cost. We derive analytical conditions for which an interference scheme or strategy can guarantee a given level of cooperation while at the same time minimising the total cost of investment (for rewarding cooperative behaviours), and show that the results are highly sensitive to the intensity of selection by interference. Interestingly, we show that a simple class of interference that makes investment decisions based on the population composition can lead to significantly more cost-efficient outcomes than standard institutional incentive strategies, especially in the case of weak selection.

Evolution reinforces cooperation with the emergence of self-recognition mechanisms: An empirical study of strategies in the Moran process for the iterated prisoner's dilemma

We present insights and empirical results from an extensive numerical study of the evolutionary dynamics of the iterated prisoner's dilemma. Fixation probabilities for Moran processes are obtained for all pairs of 164 different strategies including classics such as TitForTat, zero determinant strategies, and many more sophisticated strategies. Players with long memories and sophisticated behaviours outperform many strategies that perform well in a two player setting. Moreover we introduce several strategies trained with evolutionary algorithms to excel at the Moran process. These strategies are excellent invaders and resistors of invasion and in some cases naturally evolve handshaking mechanisms to resist invasion. The best invaders were those trained to maximize total payoff while the best resistors invoke handshake mechanisms. This suggests that while maximizing individual payoff can lead to the evolution of cooperation through invasion, the relatively weak invasion resistance of payoff maximizing strategies are not as evolutionarily stable as strategies employing handshake mechanisms.

Cooperation dynamics of generalized reciprocity in state-based social dilemmas

We introduce a framework for studying social dilemmas in networked societies where individuals follow a simple state-based behavioral mechanism based on generalized reciprocity, which is rooted in the principle "help anyone if helped by someone." Within this general framework, which applies to a wide range of social dilemmas including, among others, public goods, donation, and snowdrift games, we study the cooperation dynamics on a variety of complex network examples. By interpreting the studied model through the lenses of nonlinear dynamical systems, we show that cooperation through generalized reciprocity always emerges as the unique attractor in which the overall level of cooperation is maximized, while simultaneously exploitation of the participating individuals is prevented. The analysis elucidates the role of the network structure, here captured by a local centrality measure which uniquely quantifies the propensity of the network structure to cooperation by dictating the degree of cooperation displayed both at the microscopic and macroscopic level. We demonstrate the applicability of the analysis on a practical example by considering an interaction structure that couples a donation process with a public goods game.

Social norm complexity and past reputations in the evolution of cooperation

Indirect reciprocity is the most elaborate and cognitively demanding of all known cooperation mechanisms, and is the most specifically human because it involves reputation and status. By helping someone, individuals may increase their reputation, which may change the predisposition of others to help them in future. The revision of an individual's reputation depends on the social norms that establish what characterizes a good or bad action and thus provide a basis for morality. Norms based on indirect reciprocity are often sufficiently complex that an individual's ability to follow subjective rules becomes important, even in models that disregard the past reputations of individuals, and reduce reputations to either 'good' or 'bad' and actions to binary decisions. Here we include past reputations in such a model and identify the key pattern in the associated norms that promotes cooperation. Of the norms that comply with this pattern, the one that leads to maximal cooperation (greater than 90 per cent) with minimum complexity does not discriminate on the basis of past reputation; the relative performance of this norm is particularly evident when we consider a 'complexity cost' in the decision process. This combination of high cooperation and low complexity suggests that simple moral principles can elicit cooperation even in complex environments.

Evolution of cooperation in stochastic games

Social dilemmas occur when incentives for individuals are misaligned with group interests^1-7. According to the 'tragedy of the commons', these misalignments can lead to overexploitation and collapse of public resources. The resulting behaviours can be analysed with the tools of game theory⁸. The theory of direct reciprocity^9-15 suggests that repeated interactions can alleviate such dilemmas, but previous work has assumed that the public resource remains constant over time. Here we introduce the idea that the public resource is instead changeable and depends on the strategic choices of individuals. An intuitive scenario is that cooperation increases the public resource, whereas defection decreases it. Thus, cooperation allows the possibility of playing a more valuable game with higher payoffs, whereas defection leads to a less valuable game. We analyse this idea using the theory of stochastic games^16-19 and evolutionary game theory. We find that the dependence of the public resource on previous interactions can greatly enhance the propensity for cooperation. For these results, the interaction between reciprocity and payoff feedback is crucial: neither repeated interactions in a constant environment nor single interactions in a changing environment yield similar cooperation rates. Our framework shows which feedbacks between exploitation and environment-either naturally occurring or designed-help to overcome social dilemmas.

Single equalizer strategy with no information transfer for conflict escalation

In an iterated two-person game, for instance prisoner's dilemma or the snowdrift game, there exist strategies that force the payoffs of the opponents to be equal. These equalizer strategies form a subset of the more general zero-determinant strategies that unilaterally set the payoff of an opponent. A challenge in the attempts to understand the role of these strategies in the evolution of animal behavior is the lack of iterations in the fights for mating opportunities or territory control. We show that an arbitrary two-parameter strategy may possess a corresponding equalizer strategy which produces the same result: statistics of the fight outcomes in the contests with competitors are the same for each of these two strategies. Therefore, analyzing only the equalizer strategy space may be sufficient to predict animal behavior if nature, indeed, reduces (marginalizes) complex strategies to equalizer strategy space. The work's main finding is that there is a unique equalizer strategy that predicts fight outcomes without symmetric cooperation responses. The lack of symmetric cooperation responses is a common trait in conflict escalation contests that generally require a clear winner. In addition, this unique strategy does not assess information of the opponent's state. The method bypasses the standard analysis of evolutionary stability. The results fit well the observations of combat between male bowl and doily spiders and support an empirical assumption of the war of attrition model that the species use only information regarding their own state during conflict escalation.

Seven rules to avoid the tragedy of the commons

Cooperation among self-interested players in a social dilemma is fragile and easily interrupted by mistakes. In this work, we study the repeated n-person public-goods game and search for a strategy that forms a cooperative Nash equilibrium in the presence of implementation error with a guarantee that the resulting payoff will be no less than any of the co-players'. By enumerating strategic possibilities for n=3, we show that such a strategy indeed exists when its memory length m equals three. It means that a deterministic strategy can be publicly employed to stabilize cooperation against error with avoiding the risk of being exploited. We furthermore show that, for general n-person public-goods game, m ≥ n is necessary to satisfy the above criteria.

Partners and rivals in direct reciprocity

Reciprocity is a major factor in human social life and accounts for a large part of cooperation in our communities. Direct reciprocity arises when repeated interactions occur between the same individuals. The framework of iterated games formalizes this phenomenon. Despite being introduced more than five decades ago, the concept keeps offering beautiful surprises. Recent theoretical research driven by new mathematical tools has proposed a remarkable dichotomy among the crucial strategies: successful individuals either act as partners or as rivals. Rivals strive for unilateral advantages by applying selfish or extortionate strategies. Partners aim to share the payoff for mutual cooperation, but are ready to fight back when being exploited. Which of these behaviours evolves depends on the environment. Whereas small population sizes and a limited number of rounds favour rivalry, partner strategies are selected when populations are large and relationships stable. Only partners allow for evolution of cooperation, while the rivals' attempt to put themselves first leads to defection.

Crosstalk in concurrent repeated games impedes direct reciprocity and requires stronger levels of forgiveness

Direct reciprocity is a mechanism for cooperation among humans. Many of our daily interactions are repeated. We interact repeatedly with our family, friends, colleagues, members of the local and even global community. In the theory of repeated games, it is a tacit assumption that the various games that a person plays simultaneously have no effect on each other. Here we introduce a general framework that allows us to analyze “crosstalk” between a player’s concurrent games. In the presence of crosstalk, the action a person experiences in one game can alter the person’s decision in another. We find that crosstalk impedes the maintenance of cooperation and requires stronger levels of forgiveness. The magnitude of the effect depends on the population structure. In more densely connected social groups, crosstalk has a stronger effect. A harsh retaliator, such as Tit-for-Tat, is unable to counteract crosstalk. The crosstalk framework provides a unified interpretation of direct and upstream reciprocity in the context of repeated games.

Social interactions among people are often repeated, and yet it is assumed that simultaneous interactions are independent from one another. Here, Reiter and colleagues describe a conceptual framework where an action in one game can influence the decision in another.

Symmetric Decomposition of Asymmetric Games

We introduce new theoretical insights into two-population asymmetric games allowing for an elegant symmetric decomposition into two single population symmetric games. Specifically, we show how an asymmetric bimatrix game (A,B) can be decomposed into its symmetric counterparts by envisioning and investigating the payoff tables (A and B) that constitute the asymmetric game, as two independent, single population, symmetric games. We reveal several surprising formal relationships between an asymmetric two-population game and its symmetric single population counterparts, which facilitate a convenient analysis of the original asymmetric game due to the dimensionality reduction of the decomposition. The main finding reveals that if (x,y) is a Nash equilibrium of an asymmetric game (A,B), this implies that y is a Nash equilibrium of the symmetric counterpart game determined by payoff table A, and x is a Nash equilibrium of the symmetric counterpart game determined by payoff table B. Also the reverse holds and combinations of Nash equilibria of the counterpart games form Nash equilibria of the asymmetric game. We illustrate how these formal relationships aid in identifying and analysing the Nash structure of asymmetric games, by examining the evolutionary dynamics of the simpler counterpart games in several canonical examples.

Chaos and unpredictability in evolution of cooperation in continuous time

Cooperators benefit others with paying costs. Evolution of cooperation crucially depends on the cost-benefit ratio of cooperation, denoted as c. In this work, we investigate the infinitely repeated prisoner's dilemma for various values of c with four of the representative memory-one strategies, i.e., unconditional cooperation, unconditional defection, tit-for-tat, and win-stay-lose-shift. We consider replicator dynamics which deterministically describes how the fraction of each strategy evolves over time in an infinite-sized well-mixed population in the presence of implementation error and mutation among the four strategies. Our finding is that this three-dimensional continuous-time dynamics exhibits chaos through a bifurcation sequence similar to that of a logistic map as c varies. If mutation occurs with rate μ≪1, the position of the bifurcation sequence on the c axis is numerically found to scale as μ^{0.1}, and such sensitivity to μ suggests that mutation may have nonperturbative effects on evolutionary paths. It demonstrates how the microscopic randomness of the mutation process can be amplified to macroscopic unpredictability by evolutionary dynamics.