Collapse of cooperation in evolving games

Empirical evidence of resource dependent evolution of payoff matrices in Saccharomyces cerevisiae populations

In evolutionary game theory, a relative comparison of the cost and benefit associated with obtaining a resource, called payoff, is used as an indicator of fitness of an organism. Payoffs of different strategies, quantitatively represented as payoff matrices, are used to understand complex inter-species and intra-species interactions like cooperation, mutualism, and altruism. Payoff matrices, however, are usually treated as invariant with time-largely due to the absence of any empirical data quantifying their evolution. In this paper, we present empirical evidence of three types of resource-dependent changes in the payoff matrices of evolving Saccharomyces cerevisiae populations. We show that depending on the carbon source and participating genotypes, N-player games could collapse, be born, or be maintained. Our results highlight the need to consider the dynamic nature of payoff matrices while making even short-term predictions about population interactions and dynamics.

Conditional cooperation with longer memory

Direct reciprocity is a wide-spread mechanism for the evolution of cooperation. In repeated interactions, players can condition their behavior on previous outcomes. A well-known approach is given by reactive strategies, which respond to the coplayer's previous move. Here, we extend reactive strategies to longer memories. A reactive-n strategy takes into account the sequence of the last n moves of the coplayer. A reactive-n counting strategy responds to how often the coplayer cooperated during the last n rounds. We derive an algorithm to identify the partner strategies within these strategy sets. Partner strategies are those that ensure mutual cooperation without exploitation. We give explicit conditions for all partner strategies among reactive-2, reactive-3 strategies, and reactive-n counting strategies. To further explore the role of memory, we perform evolutionary simulations. We vary several key parameters, such as the cost-to-benefit ratio of cooperation, the error rate, and the strength of selection. Within the strategy sets we consider, we find that longer memory tends to promote cooperation. This positive effect of memory is particularly pronounced when individuals take into account the precise sequence of moves.

Evolution of cooperation in multichannel games on multiplex networks

Humans navigate diverse social relationships and concurrently interact across multiple social contexts. An individual's behavior in one context can influence behavior in other contexts. Different payoffs associated with interactions in the different domains have motivated recent studies of the evolution of cooperation through the analysis of multichannel games where each individual is simultaneously engaged in multiple repeated games. However, previous investigations have ignored the potential role of network structure in each domain and the effect of playing against distinct interacting partners in different domains. Multiplex networks provide a useful framework to represent social interactions between the same set of agents across different social contexts. We investigate the role of multiplex network structure and strategy linking in multichannel games on the spread of cooperative behavior in all layers of the multiplex. We find that multiplex structure along with strategy linking enhances the cooperation rate in all layers of the multiplex compared to a well-mixed population in Prisoners' Dilemma games, provided the network structure is identical across layers. The effectiveness of strategy linking in enhancing cooperation depends on the degree of similarity of the network structure across the layers and perception errors due to imperfect memory. Higher cooperation rates are achieved when the degree of structural overlap of the different layers is sufficiently large, and the probability of perception error is relatively low. Our work reveals how the social network structure in different layers of a multiplex can affect the spread of cooperation by limiting the ability of individuals to link strategies across different social domains.

Evolution of reciprocity with limited payoff memory

Direct reciprocity is a mechanism for the evolution of cooperation in repeated social interactions. According to the literature, individuals naturally learn to adopt conditionally cooperative strategies if they have multiple encounters with their partner. Corresponding models have greatly facilitated our understanding of cooperation, yet they often make strong assumptions on how individuals remember and process payoff information. For example, when strategies are updated through social learning, it is commonly assumed that individuals compare their average payoffs. This would require them to compute (or remember) their payoffs against everyone else in the population. To understand how more realistic constraints influence direct reciprocity, we consider the evolution of conditional behaviours when individuals learn based on more recent experiences. Even in the most extreme case that they only take into account their very last interaction, we find that cooperation can still evolve. However, such individuals adopt less generous strategies, and they cooperate less often than in the classical setup with average payoffs. Interestingly, once individuals remember the payoffs of two or three recent interactions, cooperation rates quickly approach the classical limit. These findings contribute to a literature that explores which kind of cognitive capabilities are required for reciprocal cooperation. While our results suggest that some rudimentary form of payoff memory is necessary, it suffices to remember a few interactions.

Efficiency and resilience of cooperation in asymmetric social dilemmas

Direct reciprocity is a powerful mechanism for cooperation in social dilemmas. The very logic of reciprocity, however, seems to require that individuals are symmetric, and that everyone has the same means to influence each others' payoffs. Yet in many applications, individuals are asymmetric. Herein, we study the effect of asymmetry in linear public good games. Individuals may differ in their endowments (their ability to contribute to a public good) and in their productivities (how effective their contributions are). Given the individuals' productivities, we ask which allocation of endowments is optimal for cooperation. To this end, we consider two notions of optimality. The first notion focuses on the resilience of cooperation. The respective endowment distribution ensures that full cooperation is feasible even under the most adverse conditions. The second notion focuses on efficiency. The corresponding endowment distribution maximizes group welfare. Using analytical methods, we fully characterize these two endowment distributions. This analysis reveals that both optimality notions favor some endowment inequality: More productive players ought to get higher endowments. Yet the two notions disagree on how unequal endowments are supposed to be. A focus on resilience results in less inequality. With additional simulations, we show that the optimal endowment allocation needs to account for both the resilience and the efficiency of cooperation.

The distorting effects of producer strategies: Why engagement does not reveal consumer preferences for misinformation

A great deal of empirical research has examined who falls for misinformation and why. Here, we introduce a formal game-theoretic model of engagement with news stories that captures the strategic interplay between (mis)information consumers and producers. A key insight from the model is that observed patterns of engagement do not necessarily reflect the preferences of consumers. This is because producers seeking to promote misinformation can use strategies that lead moderately inattentive readers to engage more with false stories than true ones-even when readers prefer more accurate over less accurate information. We then empirically test people's preferences for accuracy in the news. In three studies, we find that people strongly prefer to click and share news they perceive as more accurate-both in a general population sample, and in a sample of users recruited through Twitter who had actually shared links to misinformation sites online. Despite this preference for accurate news-and consistent with the predictions of our model-we find markedly different engagement patterns for articles from misinformation versus mainstream news sites. Using 1,000 headlines from 20 misinformation and 20 mainstream news sites, we compare Facebook engagement data with 20,000 accuracy ratings collected in a survey experiment. Engagement with a headline is negatively correlated with perceived accuracy for misinformation sites, but positively correlated with perceived accuracy for mainstream sites. Taken together, these theoretical and empirical results suggest that consumer preferences cannot be straightforwardly inferred from empirical patterns of engagement.

Can institutions foster cooperation by wealth redistribution?

Theoretical models prescribe how institutions can promote cooperation in a population by imposing appropriate punishments or rewards on individuals. However, many real-world institutions are not sophisticated or responsive enough to ensure cooperation by calibrating their policies. Or, worse yet, an institution might selfishly exploit the population it governs for its own benefit. Here, we study the evolution of cooperation in the presence of an institution that is autonomous, in the sense that it has its own interests that may or may not align with those of the population. The institution imposes a tax on the population and redistributes a portion of the tax revenue to cooperators, withholding the remaining revenue for itself. The institution adjusts its rates of taxation and redistribution to optimize its own long-term, discounted utility. We consider three types of institutions with different goals, embodied in their utility functions. We show that a prosocial institution, whose goal is to maximize the average payoff of the population, can indeed promote cooperation-but only if it is sufficiently forward-looking. On the other hand, an institution that seeks to maximize welfare among cooperators alone will successfully promote collective cooperation even if it is myopic. Remarkably, even a selfish institution, which seeks to maximize the revenue it withholds for itself, can nonetheless promote cooperation. The average payoff of the population increases when a selfish institution is more forward-looking, so that a population under a selfish regime can sometimes fare better than under anarchy. Our analysis highlights the potential benefits of institutional wealth redistribution, even when an institution does not share the interests of the population it governs.

Evolution of a theory of mind

Even though the Theory of Mind in upper primates has been under investigation for decades, how it may evolve remains an open problem. We propose here an evolutionary game theoretical model where a finite population of individuals may use reasoning strategies to infer a response to the anticipated behavior of others within the context of a sequential dilemma, i.e., the Centipede Game. We show that strategies with bounded reasoning evolve and flourish under natural selection, provided they are allowed to make reasoning mistakes and a temptation for higher future gains is in place. We further show that non-deterministic reasoning co-evolves with an optimism bias that may lead to the selection of new equilibria, closely associated with average behavior observed in experimental data. This work reveals both a novel perspective on the evolution of bounded rationality and a co-evolutionary link between the evolution of Theory of Mind and the emergence of misbeliefs.

Trojan horselike T6SS effector TepC mediates both interference competition and exploitative competition

The type VI secretion system (T6SS) is a bacterial weapon capable of delivering antibacterial effectors to kill competing cells for interference competition, as well as secreting metal ion scavenging effectors to acquire essential micronutrients for exploitation competition. However, no T6SS effectors that can mediate both interference competition and exploitation competition have been reported. In this study, we identified a unique T6SS-1 effector in Yersinia pseudotuberculosis named TepC, which plays versatile roles in microbial communities. First, secreted TepC acts as a proteinaceous siderophore that binds to iron and mediates exploitative competition. Additionally, we discovered that TepC has DNase activity, which gives it both contact-dependent and contact-independent interference competition abilities. In conditions where iron is limited, the iron-loaded TepC is taken up by target cells expressing the outer membrane receptor TdsR. For kin cells encoding the cognate immunity protein TipC, TepC facilitates iron acquisition, and its toxic effects are neutralized. On the other hand, nonkin cells lacking TipC are enticed to uptake TepC and are killed by its DNase activity. Therefore, we have uncovered a T6SS effector, TepC, that functions like a "Trojan horse" by binding to iron ions to provide a valuable resource to kin cells, whereas punishing cheaters that do not produce public goods. This lure-to-kill mechanism, mediated by a bifunctional T6SS effector, may offer new insights into the molecular mechanisms that maintain stability in microbial communities.

The effect of environmental information on evolution of cooperation in stochastic games

Many human interactions feature the characteristics of social dilemmas where individual actions have consequences for the group and the environment. The feedback between behavior and environment can be studied with the framework of stochastic games. In stochastic games, the state of the environment can change, depending on the choices made by group members. Past work suggests that such feedback can reinforce cooperative behaviors. In particular, cooperation can evolve in stochastic games even if it is infeasible in each separate repeated game. In stochastic games, participants have an interest in conditioning their strategies on the state of the environment. Yet in many applications, precise information about the state could be scarce. Here, we study how the availability of information (or lack thereof) shapes evolution of cooperation. Already for simple examples of two state games we find surprising effects. In some cases, cooperation is only possible if there is precise information about the state of the environment. In other cases, cooperation is most abundant when there is no information about the state of the environment. We systematically analyze all stochastic games of a given complexity class, to determine when receiving information about the environment is better, neutral, or worse for evolution of cooperation.

Grouping promotes both partnership and rivalry with long memory in direct reciprocity

Biological and social scientists have long been interested in understanding how to reconcile individual and collective interests in the iterated Prisoner's Dilemma. Many effective strategies have been proposed, and they are often categorized into one of two classes, 'partners' and 'rivals.' More recently, another class, 'friendly rivals,' has been identified in longer-memory strategy spaces. Friendly rivals qualify as both partners and rivals: They fully cooperate with themselves, like partners, but never allow their co-players to earn higher payoffs, like rivals. Although they have appealing theoretical properties, it is unclear whether they would emerge in an evolving population because most previous works focus on the memory-one strategy space, where no friendly rival strategy exists. To investigate this issue, we have conducted evolutionary simulations in well-mixed and group-structured populations and compared the evolutionary dynamics between memory-one and longer-memory strategy spaces. In a well-mixed population, the memory length does not make a major difference, and the key factors are the population size and the benefit of cooperation. Friendly rivals play a minor role because being a partner or a rival is often good enough in a given environment. It is in a group-structured population that memory length makes a stark difference: When longer-memory strategies are available, friendly rivals become dominant, and the cooperation level nearly reaches a maximum, even when the benefit of cooperation is so low that cooperation would not be achieved in a well-mixed population. This result highlights the important interaction between group structure and memory lengths that drive the evolution of cooperation.

Adaptive dynamics of memory-one strategies in the repeated donation game

Human interactions can take the form of social dilemmas: collectively, people fare best if all cooperate but each individual is tempted to free ride. Social dilemmas can be resolved when individuals interact repeatedly. Repetition allows them to adopt reciprocal strategies which incentivize cooperation. The most basic model for direct reciprocity is the repeated donation game, a variant of the prisoner's dilemma. Two players interact over many rounds; in each round they decide whether to cooperate or to defect. Strategies take into account the history of the play. Memory-one strategies depend only on the previous round. Even though they are among the most elementary strategies of direct reciprocity, their evolutionary dynamics has been difficult to study analytically. As a result, much previous work has relied on simulations. Here, we derive and analyze their adaptive dynamics. We show that the four-dimensional space of memory-one strategies has an invariant three-dimensional subspace, generated by the memory-one counting strategies. Counting strategies record how many players cooperated in the previous round, without considering who cooperated. We give a partial characterization of adaptive dynamics for memory-one strategies and a full characterization for memory-one counting strategies.

Coevolutionary dynamics via adaptive feedback in collective-risk social dilemma game

Human society and natural environment form a complex giant ecosystem, where human activities not only lead to the change in environmental states, but also react to them. By using collective-risk social dilemma game, some studies have already revealed that individual contributions and the risk of future losses are inextricably linked. These works, however, often use an idealistic assumption that the risk is constant and not affected by individual behaviors. Here, we develop a coevolutionary game approach that captures the coupled dynamics of cooperation and risk. In particular, the level of contributions in a population affects the state of risk, while the risk in turn influences individuals' behavioral decision-making. Importantly, we explore two representative feedback forms describing the possible effect of strategy on risk, namely, linear and exponential feedbacks. We find that cooperation can be maintained in the population by keeping at a certain fraction or forming an evolutionary oscillation with risk, independently of the feedback type. However, such evolutionary outcome depends on the initial state. Taken together, a two-way coupling between collective actions and risk is essential to avoid the tragedy of the commons. More importantly, a critical starting portion of cooperators and risk level is what we really need for guiding the evolution toward a desired direction.

Mutation enhances cooperation in direct reciprocity

Direct reciprocity is a powerful mechanism for the evolution of cooperation based on repeated interactions between the same individuals. But high levels of cooperation evolve only if the benefit-to-cost ratio exceeds a certain threshold that depends on memory length. For the best-explored case of one-round memory, that threshold is two. Here, we report that intermediate mutation rates lead to high levels of cooperation, even if the benefit-to-cost ratio is only marginally above one, and even if individuals only use a minimum of past information. This surprising observation is caused by two effects. First, mutation generates diversity which undermines the evolutionary stability of defectors. Second, mutation leads to diverse communities of cooperators that are more resilient than homogeneous ones. This finding is relevant because many real-world opportunities for cooperation have small benefit-to-cost ratios, which are between one and two, and we describe how direct reciprocity can attain cooperation in such settings. Our result can be interpreted as showing that diversity, rather than uniformity, promotes evolution of cooperation.

Investigating the eco-evolutionary tunnels for establishing cooperative communities

Diversity is abundant among microbial communities. Understanding the assembly of diverse microbial communities is a significant challenge. One of the recent plausible explanations for the assembly involves eco-evolutionary tunnels, where species interact in the same timescale with the mutational rate. Analysis of data generated by agent-based models was used to understand these tunnels. However, modeling the interactions explicitly by dynamic models is lacking. Here, we present the modeling and characterization of eco-evolutionary tunnels that give rise to cooperative evolutionary stable communities (ESC). We find that higher order, but common interactions are sufficient for eco-evolutionary tunnels. We identify three distinct scenarios: evolution of costly cooperation, mutationally inaccessible assembly, and bistability. Biological interpretations of the models are shedding light on the evolution of cooperation. One of the important findings is that if species maximize their benefit by preying on the other strain when dominant and cooperating at intermediate abundances, the assembly process needs eco-evolutionary tunneling. In addition, we characterize the importance of genetic drift with respect to eco-evolutionary tunnels, intermittently stable communities, and the effect of high population limits on the tunnels.

Generic catastrophic poverty when selfish investors exploit a degradable common resource

The productivity of a common pool of resources may degrade when overly exploited by a number of selfish investors, a situation known as the tragedy of the commons. Without regulations, agents optimize the size of their individual investments into the commons by balancing incurring costs with the returns received. The resulting Nash equilibrium involves a self-consistency loop between individual investment decisions and the state of the commons. As a consequence, several non-trivial properties emerge. For N investing actors we prove rigorously that typical payoffs do not scale as 1/N, the expected result for cooperating agents, but as (1/N)². Payoffs are hence reduced with regard to the functional dependence on N, a situation denoted catastrophic poverty. We show that catastrophic poverty results from a fine-tuned balance between returns and costs. Additionally, a finite number of oligarchs may be present. Oligarchs are characterized by payoffs that are finite and not decreasing when N increases. Our results hold for generic classes of models, including convex and moderately concave cost functions. For strongly concave cost functions the Nash equilibrium undergoes a collective reorganization, being characterized instead by entry barriers and sudden death forced market exits.

Group reciprocity and the evolution of stereotyping

Stereotypes are generalized beliefs about groups of people, which are used to make decisions and judgements about them. Although such heuristics can be useful when decisions must be made quickly, or when information is lacking, they can also serve as the basis for prejudice and discrimination. In this paper, we study the evolution of stereotypes through group reciprocity. We characterize the warmth of a stereotype as the willingness to cooperate with an individual based solely on the identity of the group they belong to. We show that when stereotype groups are large, such group reciprocity is less likely to evolve, and stereotypes tend to be negative. We also show that, even when stereotypes are broadly positive, individuals are often overly pessimistic about the willingness of those they stereotype to cooperate. We then show that the tendency for stereotyping itself to evolve is driven by the costs of cognition, so that more people are stereotyped with greater coarseness as costs increase. Finally we show that extrinsic 'shocks', in which the benefits of cooperation are suddenly reduced, can cause stereotype warmth and judgement bias to turn sharply negative, consistent with the view that economic and other crises are drivers of out-group animosity.

Evolution of direct reciprocity in group-structured populations

People tend to have their social interactions with members of their own community. Such group-structured interactions can have a profound impact on the behaviors that evolve. Group structure affects the way people cooperate, and how they reciprocate each other's cooperative actions. Past work has shown that population structure and reciprocity can both promote the evolution of cooperation. Yet the impact of these mechanisms has been typically studied in isolation. In this work, we study how the two mechanisms interact. Using a game-theoretic model, we explore how people engage in reciprocal cooperation in group-structured populations, compared to well-mixed populations of equal size. In this model, the population is subdivided into groups. Individuals engage in pairwise interactions within groups while they also have chances to imitate strategies outside the groups. To derive analytical results, we focus on two scenarios. In the first scenario, we assume a complete separation of time scales. Mutations are rare compared to between-group comparisons, which themselves are rare compared to within-group comparisons. In the second scenario, there is a partial separation of time scales, where mutations and between-group comparisons occur at a comparable rate. In both scenarios, we find that the effect of population structure depends on the benefit of cooperation. When this benefit is small, group-structured populations are more cooperative. But when the benefit is large, well-mixed populations result in more cooperation. Overall, our results reveal how group structure can sometimes enhance and sometimes suppress the evolution of cooperation.

Evolution of cooperation through cumulative reciprocity

Reciprocity is a simple principle for cooperation that explains many of the patterns of how humans seek and receive help from each other. To capture reciprocity, traditional models often assume that individuals use simple strategies with restricted memory. These memory-1 strategies are mathematically convenient, but they miss important aspects of human reciprocity, where defections can have lasting effects. Here we instead propose a strategy of cumulative reciprocity. Cumulative reciprocators count the imbalance of cooperation across their previous interactions with their opponent. They cooperate as long as this imbalance is sufficiently small. Using analytical and computational methods, we show that this strategy can sustain cooperation in the presence of errors, that it enforces fair outcomes and that it evolves in hostile environments. Using an economic experiment, we confirm that cumulative reciprocity is more predictive of human behaviour than several classical strategies. The basic principle of cumulative reciprocity is versatile and can be extended to a range of social dilemmas.

The conditional defector strategies can violate the most crucial supporting mechanisms of cooperation

Cooperation is essential for all domains of life. Yet, ironically, it is intrinsically vulnerable to exploitation by cheats. Hence, an explanatory necessity spurs many evolutionary biologists to search for mechanisms that could support cooperation. In general, cooperation can emerge and be maintained when cooperators are sufficiently interacting with themselves. This communication provides a kind of assortment and reciprocity. The most crucial and common mechanisms to achieve that task are kin selection, spatial structure, and enforcement (punishment). Here, we used agent-based simulation models to investigate these pivotal mechanisms against conditional defector strategies. We concluded that the latter could easily violate the former and take over the population. This surprising outcome may urge us to rethink the evolution of cooperation, as it illustrates that maintaining cooperation may be more difficult than previously thought. Moreover, empirical applications may support these theoretical findings, such as invading the cooperator population of pathogens by genetically engineered conditional defectors, which could be a potential therapy for many incurable diseases.

Evolutionary instability of selfish learning in repeated games

Across many domains of interaction, both natural and artificial, individuals use past experience to shape future behaviors. The results of such learning processes depend on what individuals wish to maximize. A natural objective is one's own success. However, when two such "selfish" learners interact with each other, the outcome can be detrimental to both, especially when there are conflicts of interest. Here, we explore how a learner can align incentives with a selfish opponent. Moreover, we consider the dynamics that arise when learning rules themselves are subject to evolutionary pressure. By combining extensive simulations and analytical techniques, we demonstrate that selfish learning is unstable in most classical two-player repeated games. If evolution operates on the level of long-run payoffs, selection instead favors learning rules that incorporate social (other-regarding) preferences. To further corroborate these results, we analyze data from a repeated prisoner's dilemma experiment. We find that selfish learning is insufficient to explain human behavior when there is a trade-off between payoff maximization and fairness.

Prisoner's Dilemma Game with Cooperation-Defection Dominance Strategies on Correlational Multilayer Networks

As multilayer networks are widely applied in modern society, numerous studies have shown the impact of a multilayer network structure and the network nature on the proportion of cooperators in the network. In this paper, we use Barabási-Albert scale-free networks (BA) and Watts and Strogatz networks (WS) to build a multilayer network structure, and we propose a new strategy-updating rule called "cooperation-defection dominance", which can be likened to dominant and recessive traits in biogenetics. With the newly constructed multilayer network structure and the strategy-updating rules, based on the simulation results, we find that in the BA-BA network, the cooperation dominance strategy can make the networks with different rs show a cooperative trend, while the defection dominance strategy only has an obvious effect on the network cooperation with a larger r. When the BA network is connected to the WS network, we find that the effect of strategy on the proportion of cooperators in the network decreases, and the main influencing factor is the structure of the network. In the three-layer network, the cooperation dominance strategy has a greater impact on the BA network, and the proportion of the cooperators is enhanced more than under the natural evolution strategy, but the promotion effect is still smaller than that of the two-layer BA network because of the WS network. Under the defection dominance strategy, the WS layer appears different from the first two strategies, and we conclude through simulation that when the payoff parameter is at the middle level, its cooperator proportion will be suppressed, and we deduce that the proportion of cooperators and defectors, as well as the payoff, play an important role.

Direct reciprocity between individuals that use different strategy spaces

In repeated interactions, players can use strategies that respond to the outcome of previous rounds. Much of the existing literature on direct reciprocity assumes that all competing individuals use the same strategy space. Here, we study both learning and evolutionary dynamics of players that differ in the strategy space they explore. We focus on the infinitely repeated donation game and compare three natural strategy spaces: memory-1 strategies, which consider the last moves of both players, reactive strategies, which respond to the last move of the co-player, and unconditional strategies. These three strategy spaces differ in the memory capacity that is needed. We compute the long term average payoff that is achieved in a pairwise learning process. We find that smaller strategy spaces can dominate larger ones. For weak selection, unconditional players dominate both reactive and memory-1 players. For intermediate selection, reactive players dominate memory-1 players. Only for strong selection and low cost-to-benefit ratio, memory-1 players dominate the others. We observe that the supergame between strategy spaces can be a social dilemma: maximum payoff is achieved if both players explore a larger strategy space, but smaller strategy spaces dominate.

Cooperation in alternating interactions with memory constraints

In repeated social interactions, individuals often employ reciprocal strategies to maintain cooperation. To explore the emergence of reciprocity, many theoretical models assume synchronized decision making. In each round, individuals decide simultaneously whether to cooperate or not. Yet many manifestations of reciprocity in nature are asynchronous. Individuals provide help at one time and receive help at another. Here, we explore such alternating games in which players take turns. We mathematically characterize all Nash equilibria among memory-one strategies. Moreover, we use evolutionary simulations to explore various model extensions, exploring the effect of discounted games, irregular alternation patterns, and higher memory. In all cases, we observe that mutual cooperation still evolves for a wide range of parameter values. However, compared to simultaneous games, alternating games require different strategies to maintain cooperation in noisy environments. Moreover, none of the respective strategies are evolutionarily stable.

In many instances of reciprocity, individuals cooperate in turns. Here, the authors analyze the equilibria and the dynamics of such alternating games, and in particular describe all strategies with one-round memory that maintain cooperation.

Collective self-optimization of communicating active particles

The quest for how to collectively self-organize in order to maximize the survival chances of the members of a social group requires finding an optimal compromise between maximizing the well-being of an individual and that of the group. Here we develop a minimal model describing active individuals which consume or produce, and respond to a shared resource-such as the oxygen concentration for aerotactic bacteria or the temperature field for penguins-while urging for an optimal resource value. Notably, this model can be approximated by an attraction-repulsion model, but, in general, it features many-body interactions. While the former prevents some individuals from closely approaching the optimal value of the shared "resource field," the collective many-body interactions induce aperiodic patterns, allowing the group to collectively self-optimize. Arguably, the proposed optimal field-based collective interactions represent a generic concept at the interface of active matter physics, collective behavior, and microbiological chemotaxis. This concept might serve as a useful ingredient to optimize ensembles of synthetic active agents or to help unveil aspects of the communication rules which certain social groups use to maximize their survival chances.

A unified framework of direct and indirect reciprocity

Direct and indirect reciprocity are key mechanisms for the evolution of cooperation. Direct reciprocity means that individuals use their own experience to decide whether to cooperate with another person. Indirect reciprocity means that they also consider the experiences of others. Although these two mechanisms are intertwined, they are typically studied in isolation. Here, we introduce a mathematical framework that allows us to explore both kinds of reciprocity simultaneously. We show that the well-known 'generous tit-for-tat' strategy of direct reciprocity has a natural analogue in indirect reciprocity, which we call 'generous scoring'. Using an equilibrium analysis, we characterize under which conditions either of the two strategies can maintain cooperation. With simulations, we additionally explore which kind of reciprocity evolves when members of a population engage in social learning to adapt to their environment. Our results draw unexpected connections between direct and indirect reciprocity while highlighting important differences regarding their evolvability.

Human players manage to extort more than the mutual cooperation payoff in repeated social dilemmas

Social dilemmas are mixed-motive games. Although the players have a common interest in maintaining cooperation, each may try to obtain a larger payoff by cooperating less than the other. This phenomenon received increased attention after Press and Dyson discovered a class of strategies for the repeated prisoner's dilemma (extortionate strategies) that secure for themselves a payoff that is never smaller, but can be larger, than the opponent's payoff. We conducted an experiment to test whether humans adopt extortionate strategies when playing a social dilemma. Our results reveal that human subjects do try to extort a larger payoff from their opponents. However, they are only successful when extortionate strategies are part of a Nash equilibrium. In settings where extortionate strategies do not appear in any Nash equilibrium, attempts at extortion only result in a breakdown of cooperation. Our subjects recognized the different incentives implied by the two settings, and they were ready to "extort" the opponent when allowed to do so. This suggests that deviations from mutually cooperative equilibria, which are usually attributed to players' impatience, coordination problems, or lack of information, can instead be driven by subjects trying to reach more favorable outcomes.

Bycatch levies could reconcile trade-offs between blue growth and biodiversity conservation

Economic activities in the ocean (that is, the 'blue economy') provide value to society, yet also jeopardize marine ecosystems. For example, fisheries are an essential source of income and food security for billions of people, yet bycatch poses a major threat to marine biodiversity, creating trade-offs between economic growth and biodiversity conservation. This Perspective explores bycatch levies as a market-based instrument for reconciling these trade-offs. We outline the theory and practice of bycatch levies to demonstrate how they could incentivize bycatch prevention and raise revenue for compensatory conservation, provided they are well designed, as part of a policy mix for sustainable and equitable ocean governance. We then explore ways forward for mainstreaming bycatch levies into the blue economy. While compensatory bycatch mitigation has been controversial, increasing adoption of net outcome approaches to biodiversity conservation suggests they could become mainstreamed within the next decade. Bycatch levies could raise billions of dollars towards closing global biodiversity financing gaps, delivering net outcomes for biodiversity under the United Nations Post-2020 Global Biodiversity Framework while enabling blue growth, and moving towards win-wins for economic welfare and biodiversity conservation.

Arbitration supports reciprocity when there are frequent perception errors

Reciprocity is undermined by perception errors, mistakes that cause disagreement between interacting individuals about past behaviour. Strategies such as win-stay-lose-shift and generous tit-for-tat can re-establish cooperation following a perception error, but only when errors arise infrequently. We introduce arbitration tit-for-tat (ATFT), a strategy that uses third-party arbitration to align players' beliefs about what transpired when they disagree. We show that, when arbitration is moderately accurate, ATFT is a strong subgame-perfect equilibrium and is evolutionarily stable against a range of strategies that defect, cooperate, ignore arbitration or invoke arbitration unnecessarily. ATFT can persist when perception errors are frequent, arbitration is costly or arbitration is biased. The need for third parties to resolve perception errors could explain why reciprocity is rare in other animals despite opportunities for repeated interactions and why human reciprocity is embedded within culturally transmitted moral norms in which community monitoring plays a role.

Avoiding the bullies: The resilience of cooperation among unequals

Can egalitarian norms or conventions survive the presence of dominant individuals who are ensured of victory in conflicts? We investigate the interaction of power asymmetry and partner choice in games of conflict over a contested resource. Previous models of cooperation do not include both power inequality and partner choice. Furthermore, models that do include power inequalities assume a static game where a bully's advantage does not change. They have therefore not attempted to model complex and realistic properties of social interaction. Here, we introduce three models to study the emergence and resilience of cooperation among unequals when interaction is random, when individuals can choose their partners, and where power asymmetries dynamically depend on accumulated payoffs. We find that the ability to avoid bullies with higher competitive ability afforded by partner choice mostly restores cooperative conventions and that the competitive hierarchy never forms. Partner choice counteracts the hyper dominance of bullies who are isolated in the network and eliminates the need for others to coordinate in a coalition. When competitive ability dynamically depends on cumulative payoffs, complex cycles of coupled network-strategy-rank changes emerge. Effective collaborators gain popularity (and thus power), adopt aggressive behavior, get isolated, and ultimately lose power. Neither the network nor behavior converge to a stable equilibrium. Despite the instability of power dynamics, the cooperative convention in the population remains stable overall and long-term inequality is completely eliminated. The interaction between partner choice and dynamic power asymmetry is crucial for these results: without partner choice, bullies cannot be isolated, and without dynamic power asymmetry, bullies do not lose their power even when isolated. We analytically identify a single critical point that marks a phase transition in all three iterations of our models. This critical point is where the first individual breaks from the convention and cycles start to emerge.

Friendly-rivalry solution to the iterated n-person public-goods game

Repeated interaction promotes cooperation among rational individuals under the shadow of future, but it is hard to maintain cooperation when a large number of error-prone individuals are involved. One way to construct a cooperative Nash equilibrium is to find a ‘friendly-rivalry’ strategy, which aims at full cooperation but never allows the co-players to be better off. Recently it has been shown that for the iterated Prisoner’s Dilemma in the presence of error, a friendly rival can be designed with the following five rules: Cooperate if everyone did, accept punishment for your own mistake, punish defection, recover cooperation if you find a chance, and defect in all the other circumstances. In this work, we construct such a friendly-rivalry strategy for the iterated n-person public-goods game by generalizing those five rules. The resulting strategy makes a decision with referring to the previous m = 2n − 1 rounds. A friendly-rivalry strategy for n = 2 inherently has evolutionary robustness in the sense that no mutant strategy has higher fixation probability in this population than that of a neutral mutant. Our evolutionary simulation indeed shows excellent performance of the proposed strategy in a broad range of environmental conditions when n = 2 and 3.

How to maintain cooperation among a number of self-interested individuals is a difficult problem, especially if they can sometimes commit error. In this work, we propose a strategy for the iterated n-person public-goods game based on the following five rules: Cooperate if everyone did, accept punishment for your own mistake, punish others’ defection, recover cooperation if you find a chance, and defect in all the other circumstances. These rules are not far from actual human behavior, and the resulting strategy guarantees three advantages: First, if everyone uses it, full cooperation is recovered even if error occurs with small probability. Second, the player of this strategy always never obtains a lower long-term payoff than any of the co-players. Third, if the co-players are unconditional cooperators, it obtains a strictly higher long-term payoff than theirs. Therefore, if everyone uses this strategy, no one has a reason to change it. Furthermore, our simulation shows that this strategy will become highly abundant over long time scales due to its robustness against the invasion of other strategies. In this sense, the repeated social dilemma is solved for an arbitrary number of players.

Competition for nutritional resources masks the true frequency of bacterial mutants

Background

It is widely assumed that all mutant microorganisms present in a culture are able to grow and form colonies, provided that they express the features required for selection. Unlike wild-type Escherichia coli, PHO-constitutive mutants overexpress alkaline phosphatase and hence can hydrolyze glycerol-2-phosphate (G2P) to glycerol and form colonies on plates having G2P as the sole carbon source. These mutations mostly occur in the pst operon. However, the frequency of PHO-constitutive colonies on the G2P selective plate is exceptionally low.

Results

We show that the rate in which spontaneous PHO-constitutive mutations emerge is about 8.0 × 10⁻⁶/generation, a relatively high rate, but the growth of most existing mutants is inhibited by their neighboring wild-type cells. This inhibition is elicited only by non-mutant viable bacteria that can take up and metabolize glycerol formed by the mutants. Evidence indicates that the few mutants that do form colonies derive from microclusters of mutants on the selective plate. A mathematical model that describes the fate of the wild-type and mutant populations under these circumstances supports these results.

Conclusion

This scenario in which neither the wild-type nor the majority of the mutants are able to grow resembles an unavoidable “tragedy of the commons” case which results in the collapse of the majority of the population. Cooperation between rare adjacent mutants enables them to overcome the competition and eventually form mutant colonies. The inhibition of PHO-constitutive mutants provides an example of mutant frequency masked by orders of magnitude due to a competition between mutants and their ancestral wild-type cells. Similar “tragedy of the commons-like” cases may occur in other settings and should be taken into consideration while estimating true mutant frequencies and mutation rates.

Supplementary Information

The online version contains supplementary material available at (doi:10.1186/s12915-020-00913-1).

Learning enables adaptation in cooperation for multi-player stochastic games

Interactions among individuals in natural populations often occur in a dynamically changing environment. Understanding the role of environmental variation in population dynamics has long been a central topic in theoretical ecology and population biology. However, the key question of how individuals, in the middle of challenging social dilemmas (e.g. the 'tragedy of the commons'), modulate their behaviours to adapt to the fluctuation of the environment has not yet been addressed satisfactorily. Using evolutionary game theory, we develop a framework of stochastic games that incorporates the adaptive mechanism of reinforcement learning to investigate whether cooperative behaviours can evolve in the ever-changing group interaction environment. When the action choices of players are just slightly influenced by past reinforcements, we construct an analytical condition to determine whether cooperation can be favoured over defection. Intuitively, this condition reveals why and how the environment can mediate cooperative dilemmas. Under our model architecture, we also compare this learning mechanism with two non-learning decision rules, and we find that learning significantly improves the propensity for cooperation in weak social dilemmas, and, in sharp contrast, hinders cooperation in strong social dilemmas. Our results suggest that in complex social-ecological dilemmas, learning enables the adaptation of individuals to varying environments.

Five rules for friendly rivalry in direct reciprocity

Direct reciprocity is one of the key mechanisms accounting for cooperation in our social life. According to recent understanding, most of classical strategies for direct reciprocity fall into one of two classes, ‘partners’ or ‘rivals’. A ‘partner’ is a generous strategy achieving mutual cooperation, and a ‘rival’ never lets the co-player become better off. They have different working conditions: For example, partners show good performance in a large population, whereas rivals do in head-to-head matches. By means of exhaustive enumeration, we demonstrate the existence of strategies that act as both partners and rivals. Among them, we focus on a human-interpretable strategy, named ‘CAPRI’ after its five characteristic ingredients, i.e., cooperate, accept, punish, recover, and defect otherwise. Our evolutionary simulation shows excellent performance of CAPRI in a broad range of environmental conditions.

Human social preferences cluster and spread in the field

While it is undeniable that the ability of humans to cooperate in large-scale societies is unique in animal life, it remains open how such a degree of prosociality is possible despite the risks of exploitation. Recent evidence suggests that social networks play a crucial role in the development of prosociality and large-scale cooperation by allowing cooperators to cluster; however, it is not well understood if and how this also applies to real-world social networks in the field. We study intrinsic social preferences alongside emerging friendship patterns in 57 freshly formed school classes (n = 1,217), using incentivized measures. We demonstrate the existence of cooperative clusters in society, examine their emergence, and expand the evidence from controlled experiments to real-world social networks. Our results suggest that being embedded in cooperative environments substantially enhances the social preferences of individuals, thus contributing to the formation of cooperative clusters. Partner choice, in contrast, only marginally contributes to their emergence. We conclude that cooperative preferences are contagious; social and cultural learning plays an important role in the development and evolution of cooperation.

Evolving cooperation in multichannel games

Humans routinely engage in many distinct interactions in parallel. Team members collaborate on several concurrent projects, and even whole nations interact with each other across a variety of issues, including trade, climate change and security. Yet the existing theory of direct reciprocity studies isolated repeated games. Such models cannot account for strategic attempts to use the vested interests in one game as a leverage to enforce cooperation in another. Here we introduce a general framework of multichannel games. Individuals interact with each other over multiple channels; each channel is a repeated game. Strategic choices in one channel can affect decisions in another. With analytical equilibrium calculations for the donation game and evolutionary simulations for several other games we show that such linkage facilitates cooperation. Our results suggest that previous studies tend to underestimate the human potential for reciprocity. When several interactions occur in parallel, people often learn to coordinate their behavior across games to maximize cooperation in each of them.

Evolutionary games with environmental feedbacks

Strategic interactions arise in all domains of life. This form of competition often plays out in dynamically changing environments. The strategies employed in a population may alter the state of the environment, which may in turn feedback to change the incentive structure of strategic interactions. Feedbacks between strategies and the environment are common in social-ecological systems, evolutionary-ecological systems, and even psychological-economic systems. Here we develop a framework of 'eco-evolutionary game theory' that enables the study of strategic and environmental dynamics with feedbacks. We consider environments governed either by intrinsic growth, decay, or tipping points. We show how the joint dynamics of strategies and the environment depend on the incentives for individuals to lead or follow behavioral changes, and on the relative speed of environmental versus strategic change. Our analysis unites dynamical phenomena that occur in settings as diverse as human decision-making, plant nutrient acquisition, and resource harvesting. We discuss implications in fields ranging from ecology to economics.

Steering eco-evolutionary game dynamics with manifold control

Feedback loops between population dynamics of individuals and their ecological environment are ubiquitously found in nature and have shown profound effects on the resulting eco-evolutionary dynamics. By incorporating linear environmental feedback law into the replicator dynamics of two-player games, recent theoretical studies have shed light on understanding the oscillating dynamics of the social dilemma. However, the detailed effects of more general nonlinear feedback loops in multi-player games, which are more common especially in microbial systems, remain unclear. Here, we focus on ecological public goods games with environmental feedbacks driven by a nonlinear selection gradient. Unlike previous models, multiple segments of stable and unstable equilibrium manifolds can emerge from the population dynamical systems. We find that a larger relative asymmetrical feedback speed for group interactions centred on cooperators not only accelerates the convergence of stable manifolds but also increases the attraction basin of these stable manifolds. Furthermore, our work offers an innovative manifold control approach: by designing appropriate switching control laws, we are able to steer the eco-evolutionary dynamics to any desired population state. Our mathematical framework is an important generalization and complement to coevolutionary game dynamics, and also fills the theoretical gap in guiding the widespread problem of population state control in microbial experiments.

Deconstructing Evolutionary Game Theory: Coevolution of Social Behaviors with Their Evolutionary Setting

Evolution of social behaviors is one of the most fascinating and active fields of evolutionary biology. During the past half century, social evolution theory developed into a mature field with powerful tools to understand the dynamics of social traits such as cooperation under a wide range of conditions. In this article, I argue that the next stage in the development of social evolution theory should consider the evolution of the setting in which social behaviors evolve. To that end, I propose a conceptual map of the components that make up the evolutionary setting of social behaviors, review existing work that considers the evolution of each component, and discuss potential future directions. The theoretical work reviewed here illustrates how unexpected dynamics can happen when the setting of social evolution itself is evolving, such as cooperation sometimes being self-limiting. I argue that a theory of how the setting of social evolution itself evolves will lead to a deeper understanding of when cooperation and other social behaviors evolve and diversify.

Evolutionary dynamics with game transitions

The environment has a strong influence on a population's evolutionary dynamics. Driven by both intrinsic and external factors, the environment is subject to continual change in nature. To capture an ever-changing environment, we consider a model of evolutionary dynamics with game transitions, where individuals' behaviors together with the games that they play in one time step influence the games to be played in the next time step. Within this model, we study the evolution of cooperation in structured populations and find a simple rule: Weak selection favors cooperation over defection if the ratio of the benefit provided by an altruistic behavior, b, to the corresponding cost, c, exceeds [Formula: see text], where k is the average number of neighbors of an individual and [Formula: see text] captures the effects of the game transitions. Even if cooperation cannot be favored in each individual game, allowing for a transition to a relatively valuable game after mutual cooperation and to a less valuable game after defection can result in a favorable outcome for cooperation. In particular, small variations in different games being played can promote cooperation markedly. Our results suggest that simple game transitions can serve as a mechanism for supporting prosocial behaviors in highly connected populations.

Wealth-relative effects in cooperation games

This paper investigates cooperation games in which poor agents do not benefit from cooperation with wealthy agents. They instead benefit from considering wealth relative to decision payoffs of fitness or wealth. Of concern is the effect of cooperation on participants, their rational self-interest and choices, and not the evolution of cooperation directly. The accumulation of fitness or wealth has been shown in the literature to lead to different optimal strategies for wealthy and poor players in Chicken games. The effect could have important explanatory power if it were more broadly applicable. First we empirically compare two published results, one involving the temptation parameter vs. degree of cooperation in Prisoner's Dilemma, and the other a surprising result from a public goods game with participants from different cultures, networks and wealth in which a fixed rather than relative payoff scheme was used. Using the temptation data to calibrate the public goods behavior suggests wealth factors can provide an explanation for the results. Second we show using simulation that adding a survival threshold to a wealth or fitness accumulating Iterated Prisoner's Dilemma produces a wealth relative effect. We clarify previous results to show the poor must avoid survival risk, regardless of whether this is associated with cooperation or defection. We do this by introducing the Farmer's Game, a simulation of Iterated Prisoner's Dilemma with wealth accumulation and a survival threshold. This is used to evaluate the Tit-for-Tat strategy and four variants. Equilibrium payoffs keep the game scaled to social relevance, with a fraction of all payoffs externalized as a turn cost parameter. Findings include poor performance of Tit-for-Tat near the survival threshold, superior performance of low risk strategies for both poor and wealthy players, dependence of survival of the poor near the threshold on Tit-for-Tat forgiveness, unexpected optimization of forgiveness without encountering a social dilemma, improved performance of a diverse mix of strategies, and a more abrupt threshold of social catastrophe for the better performing mix. Lastly we compare cooperating and non-cooperating societies using the simulation and discover disturbing connections between cooperation and familiar non-egalitarian wealth distribution patterns.

Social dilemmas among unequals

Direct reciprocity is a powerful mechanism for the evolution of cooperation on the basis of repeated interactions^1-4. It requires that interacting individuals are sufficiently equal, such that everyone faces similar consequences when they cooperate or defect. Yet inequality is ubiquitous among humans^5,6 and is generally considered to undermine cooperation and welfare^7-10. Most previous models of reciprocity do not include inequality^11-15. These models assume that individuals are the same in all relevant aspects. Here we introduce a general framework to study direct reciprocity among unequal individuals. Our model allows for multiple sources of inequality. Subjects can differ in their endowments, their productivities and in how much they benefit from public goods. We find that extreme inequality prevents cooperation. But if subjects differ in productivity, some endowment inequality can be necessary for cooperation to prevail. Our mathematical predictions are supported by a behavioural experiment in which we vary the endowments and productivities of the subjects. We observe that overall welfare is maximized when the two sources of heterogeneity are aligned, such that more productive individuals receive higher endowments. By contrast, when endowments and productivities are misaligned, cooperation quickly breaks down. Our findings have implications for policy-makers concerned with equity, efficiency and the provisioning of public goods.

Individual perception dynamics in drunk games

We study the effects of individual perceptions of payoffs in two-player games. In particular we consider the setting in which individuals' perceptions of the game are influenced by their previous experiences and outcomes. Accordingly, we introduce a framework based on evolutionary games where individuals have the capacity to perceive their interactions in different ways. Starting from the narrative of social behaviors in a pub as an illustration, we first study the combination of the Prisoner's Dilemma and Harmony Game as two alternative perceptions of the same situation. Considering a selection of game pairs, our results show that the interplay between perception dynamics and game payoffs gives rise to nonlinear phenomena unexpected in each of the games separately, such as catastrophic phase transitions in the cooperation basin of attraction, Hopf bifurcations and cycles of cooperation and defection. Combining analytical techniques with multiagent simulations, we also show how introducing individual perceptions can cause nontrivial dynamical behaviors to emerge, which cannot be obtained by analyzing the system at a macroscopic level. Specifically, initial perception heterogeneities at the microscopic level can yield a polarization effect that is unpredictable at the macroscopic level. This framework opens the door to the exploration of new ways of understanding the link between the emergence of cooperation and individual preferences and perceptions, with potential applications beyond social interactions.

Games as Tools to Address Conservation Conflicts

Conservation conflicts represent complex multilayered problems that are challenging to study. We explore the utility of theoretical, experimental, and constructivist approaches to games to help to understand and manage these challenges. We show how these approaches can help to develop theory, understand patterns in conflict, and highlight potentially effective management solutions. The choice of approach should be guided by the research question and by whether the focus is on testing hypotheses, predicting behaviour, or engaging stakeholders. Games provide an exciting opportunity to help to unravel the complexity in conflicts, while researchers need an awareness of the limitations and ethical constraints involved. Given the opportunities, this field will benefit from greater investment and development.

Evolution of Groupwise Cooperation: Generosity, Paradoxical Behavior, and Non-Linear Payoff Functions

Evolution of cooperation by reciprocity has been studied using two-player and n-player repeated prisoner’s dilemma games. An interesting feature specific to the n-player case is that players can vary in generosity, or how many defections they tolerate in a given round of a repeated game. Reciprocators are quicker to detect defectors to withdraw further cooperation when less generous, and better at maintaining a long-term cooperation in the presence of rare defectors when more generous. A previous analysis on a stochastic evolutionary model of the n-player repeated prisoner’s dilemma has shown that the fixation probability of a single reciprocator in a population of defectors can be maximized for a moderate level of generosity. However, the analysis is limited in that it considers only tit-for-tat-type reciprocators within the conventional linear payoff assumption. Here we extend the previous study by removing these limitations and show that, if the games are repeated sufficiently many times, considering non-tit-for-tat type strategies does not alter the previous results, while the introduction of non-linear payoffs sometimes does. In particular, under certain conditions, the fixation probability is maximized for a “paradoxical” strategy, which cooperates in the presence of fewer cooperating opponents than in other situations in which it defects.

Social norm complexity and past reputations in the evolution of cooperation

Indirect reciprocity is the most elaborate and cognitively demanding of all known cooperation mechanisms, and is the most specifically human because it involves reputation and status. By helping someone, individuals may increase their reputation, which may change the predisposition of others to help them in future. The revision of an individual's reputation depends on the social norms that establish what characterizes a good or bad action and thus provide a basis for morality. Norms based on indirect reciprocity are often sufficiently complex that an individual's ability to follow subjective rules becomes important, even in models that disregard the past reputations of individuals, and reduce reputations to either 'good' or 'bad' and actions to binary decisions. Here we include past reputations in such a model and identify the key pattern in the associated norms that promotes cooperation. Of the norms that comply with this pattern, the one that leads to maximal cooperation (greater than 90 per cent) with minimum complexity does not discriminate on the basis of past reputation; the relative performance of this norm is particularly evident when we consider a 'complexity cost' in the decision process. This combination of high cooperation and low complexity suggests that simple moral principles can elicit cooperation even in complex environments.