Aspiration dynamics in structured population acts as if in a well-mixed one

Cooperation among unequal players with aspiration-driven learning

Direct reciprocity promotes the evolution of cooperation when players are sufficiently equal, such that they have similar influence on each other. In the light of ubiquitous inequality, this raises the question of how reciprocity evolves among unequal players. Existing studies on inequality mainly focus on payoff-driven learning rules, which rely on the knowledge of others' strategies. However, inferring one's strategy is a difficult task even if the whole interaction history is known. Here, we consider aspiration-driven learning rules, where players seek strategies that satisfy their aspirations based on their own information. Under aspiration-driven learning rules, we explore the evolutionary dynamics among players with inequality in endowments and productivity. We model the interactions among unequal players with asymmetric games and characterize the condition where cooperation is feasible. Remarkably, we find that aspiration-driven learning rules lead to a higher level of cooperation than payoff-driven ones over a wide range of inequality. Moreover, our results show that high aspiration levels are conducive to the evolution of cooperation when more productive players are equipped with higher endowments. Our work highlights the advantages of aspiration-driven learning for promoting cooperation among unequal players and suggests that aspiration-based decision-making may be more beneficial for the collective.

Cooperation transitions in social games induced by aspiration-driven players

Cooperation and defection are social traits whose evolutionary origin is still unresolved. Recent behavioral experiments with humans suggested that strategy changes are driven mainly by the individuals' expectations and not by imitation. This work theoretically analyzes and numerically explores an aspiration-driven strategy updating in a well-mixed population playing games. The payoffs of the game matrix and the aspiration are condensed into just two parameters that allow a comprehensive description of the dynamics. We find continuous and abrupt transitions in the cooperation density with excellent agreement between theory and the Gillespie simulations. Under strong selection, the system can display several levels of steady cooperation or get trapped into absorbing states. These states are still relevant for experiments even when irrational choices are made due to their prolonged relaxation times. Finally, we show that for the particular case of the prisoner dilemma, where defection is the dominant strategy under imitation mechanisms, the self-evaluation update instead favors cooperation nonlinearly with the level of aspiration. Thus, our work provides insights into the distinct role between imitation and self-evaluation with no learning dynamics.

A game-based approach for designing a collaborative evolution mechanism for unmanned swarms on community networks

Intelligent and coordinated unmanned aerial vehicle (UAV) swarm combat will be the main mode of warfare in the future, and mechanistic design of autonomous cooperation within swarms is the key to enhancing combat effectiveness. Exploration of the essential features and patterns of autonomous collaboration in unmanned swarms has become the focus of scientific research and technological applications, in keeping with the evolving conceptions of the military theatre. However, given the unique attributes of the military and the novelty of the warfare mode of unmanned swarms, few achievements have been reported in the existing research. In this study, we analysed the military requirements of unmanned swarm operations and proposed an analytic framework for autonomous collaboration. Then, a literature review addressing swarm evolution dynamics, game-based swarm collaboration, and collaborative evolution on complex networks was conducted. Next, on the basis of the above work, we designed a community network for unmanned swarm cooperation and constructed a collaborative evolution model based on the multiplayer public goods game (PGG). Furthermore, according to the “network” and “model”, the dynamic evolution process of swarm collaboration was formally deduced. Finally, a simulation was conducted to analyse the influence of relevant parameters (i.e., swarm size, degree distribution, cost, multiplication factor) on the collaborative behaviour of unmanned swarms. According to the simulation results, some reasonable suggestions for collaborative management and control in swarm operation are given, which can provide theoretical reference and decision-making support for the design of coordination mechanisms and improved combat effectiveness in unmanned swarm operation.

Impact of the baseline payoff on evolutionary outcomes

Do individuals enjoying a higher baseline payoff behave similarly in competitive scenarios compared to their counterparts? The classical replicator equation does not answer such a question since it is invariant to the background or baseline payoff of individuals. In reality, however, if one's baseline payoff is higher than the possible payoffs of an interaction (or game), the individual may respond generously or indifferently if s(he) is satisfied with the prevailing benchmark payoff. This work intends to explore such a phenomenon within the realm of pairwise interactions-taking the prisoner's dilemma as a metaphor-in well-mixed finite and infinite populations. In this framework, a player uses the payoff (comprising baseline and game payoffs) -expectation difference to estimate a degree of eagerness and, with that degree of eagerness, revises his or her strategy with a certain probability. We adopt two approaches to explore such a context, naming them as the Fermi and imitation processes, in which the former uses a pairwise Femi function and the latter considers the relative fitness to estimate probabilities for strategy revision. In a finite population, we examine the effect of intensities to payoff-expectation and strategic payoff differences (denoted by k_{1} and k_{2}, respectively) as well as the level of contentment (ω) on the fixation probability and fixation time (for a single defector). We observe that the fixation probability surges with the increase of intensity parameters. Nevertheless, the maximum fixation probability may require a substantially larger time to fixate, especially when the expectation is lower than the baseline payoff. This means that cooperators can persist for a longer period of time. A higher expectation or greed, however, considerably reduces the fixation time. Interestingly, our numerical simulation reveals that both approaches are equivalent under weak k_{2}(≪1) in the Fermi process. We further derive mean-field equations for both approaches in the context of an infinite population, where we observe two possible evolutionary consequences: either full-scale defection or the persistence of the initial frequency of cooperators. The latter scenario indicates players' uninterested or neutral behavior in relation to the interaction due to their satisfaction on the baseline payoff.

The characteristics of average abundance function with mutation of multi-player threshold public goods evolutionary game model under redistribution mechanism

BACKGROUND

In recent years, the average abundance function has attracted much attention as it reflects the degree of cooperation in the population. Then it is significant to analyse how average abundance functions can be increased to promote the proliferation of cooperative behaviour. However, further theoretical analysis for average abundance function with mutation under redistribution mechanism is still lacking. Furthermore, the theoretical basis for the corresponding numerical simulation is not sufficiently understood.

RESULTS

We have deduced the approximate expressions of average abundance function with mutation under redistribution mechanism on the basis of different levels of selection intensity [Formula: see text] (sufficiently small and large enough). In addition, we have analysed the influence of the size of group d, multiplication factor r, cost c, aspiration level [Formula: see text] on average abundance function from both quantitative and qualitative aspects.

CONCLUSIONS

(1) The approximate expression will become the linear equation related to selection intensity when [Formula: see text] is sufficiently small. (2) On one hand, approximation expression when [Formula: see text] is large enough is not available when r is small and m is large. On the other hand, this approximation expression will become more reliable when [Formula: see text] is larger. (3) On the basis of the expected payoff function [Formula: see text] and function [Formula: see text], the corresponding results for the effects of parameters (d,r,c,[Formula: see text]) on average abundance function [Formula: see text] have been explained.

Imitation and aspiration dynamics bring different evolutionary outcomes in feedback-evolving games

Feedback-evolving games characterize the interplay between the evolution of strategies and environments. Rich dynamics have been derived for such games under the premise of the replicator equation, which unveils persistent oscillations between cooperation and defection. Besides replicator dynamics, here we have employed aspiration dynamics, in which individuals, instead of comparing payoffs with opposite strategies, assess their payoffs by self-evaluation to update strategies. We start with a brief review of feedback-evolving games with replicator dynamics and then comprehensively discuss such games with aspiration dynamics. Interestingly, the tenacious cycles, as perceived in replicator dynamics, cannot be observed in aspiration dynamics. Our analysis reveals that a parameter θ —which depicts the strength of cooperation in enhancing the environment—plays a pivotal role in comprehending the dynamics. In particular, with the symmetric aspiration level, if replete and depleted states, respectively, experience Prisoner's Dilemma and Trivial games, the rich environment is achievable only when θ  > 1. The case θ  < 1 never allows us to reach the replete state, even with a higher cooperation level. Furthermore, if cooperators aspire less than defectors, then the enhanced state can be achieved with a relatively lower θ value compared with the opposite scenario because too much expectation from cooperation can be less beneficial.

Aspiration dynamics generate robust predictions in heterogeneous populations

Update rules, which describe how individuals adjust their behavior over time, affect the outcome of social interactions. Theoretical studies have shown that evolutionary outcomes are sensitive to model details when update rules are imitation-based but are robust when update rules are self-evaluation based. However, studies of self-evaluation based rules have focused on homogeneous population structures where each individual has the same number of neighbors. Here, we consider heterogeneous population structures represented by weighted networks. Under weak selection, we analytically derive the condition for strategy success, which coincides with the classical condition of risk-dominance. This condition holds for all weighted networks and distributions of aspiration levels, and for individualized ways of self-evaluation. Our findings recover previous results as special cases and demonstrate the universality of the robustness property under self-evaluation based rules. Our work thus sheds light on the intrinsic difference between evolutionary dynamics under self-evaluation based and imitation-based update rules.

Social interaction outcomes can depend on the type of information individuals possess and how it is used in decision-making. Here, Zhou et al. find that self-evaluation based decision-making rules lead to evolutionary outcomes that are robust to different population structures and ways of self-evaluation.

Stochastic evolutionary dynamics of trust games with asymmetric parameters

Trusting in others and reciprocating that trust with trustworthy actions are crucial to successful and prosperous societies. The trust game has been widely used to quantitatively study trust and trustworthiness, involving a sequential exchange between an investor and a trustee. Deterministic evolutionary game theory predicts no trust and no trustworthiness, whereas the behavioral experiments with the one-shot anonymous trust game show that people substantially trust and respond trustworthily. To explain these discrepancies, previous works often turn to additional mechanisms, which are borrowed from other games such as the prisoner's dilemma. Although these mechanisms lead to the evolution of trust and trustworthiness to an extent, the optimal or the most common strategy often involves no trustworthiness. In this paper, we study the impact of asymmetric demographic parameters (e.g., different population sizes) on game dynamics of the trust game. We show that, in a weak-mutation limit, stochastic evolutionary dynamics with the asymmetric parameters can lead to the evolution of high trust and high trustworthiness without any additional mechanisms in well-mixed finite populations. Even full trust and near full trustworthiness can be the most common strategies. These results are qualitatively different from those of the previous works. Our results thereby demonstrate rich evolutionary dynamics of the asymmetric trust game.

Evolution of cooperation in social dilemmas under the coexistence of aspiration and imitation mechanisms

Imitation and aspiration update rules are frequently observed in human and animal populations. While the imitation process entails payoff comparisons with surroundings, the aspiration process refers to self-evaluation. This work explores the evolution of cooperation in dyadic games under the coexistence of these two dynamics in an infinitely large well-mixed population. Two situations have been explored: (i) individuals adopt either an imitation or aspiration update rule with a certain probability, and (ii) the entire population is divided into two groups where one group only uses imitative rules and the other obeys aspiration updating alone. Both premises have been modeled by taking an infinite approximation of the finite population. In particular, the second mixing principle follows an additive property: the outcome of the whole population is the weighted average of outcomes from imitators and aspiration-driven individuals. Our work progressively investigates several variants of aspiration dynamics under strong selection, encompassing symmetric, asymmetric, and adaptive aspirations, which then coalesce with imitative dynamics. We also demonstrate which of the update rules performs better, under different social dilemmas, by allowing the evolution of the preference of update rules besides strategies. Aspiration dynamics always outperform imitation dynamics in the prisoner's dilemma, however, in the chicken and stag-hunt games the predominance of either update rule depends on the level of aspirations as well as on the extent of greed and fear present in the system. Finally, we examine the coevolution of strategies, aspirations, and update rules which leads to a binary state of obeying either imitation or aspiration dynamics. In such a circumstance, when aspiration dynamics prevail over imitation dynamics, cooperators and defectors coexist to an equal extent.

Evolutionary game dynamics of combining the imitation and aspiration-driven update rules

So far, most studies on evolutionary game dynamics in finite populations have concentrated on a single update rule. However, given the impacts of the environment and individual cognition, individuals may use different update rules to change their current strategies. In light of this, the current paper reports on a study that constructed a mixed stochastic evolutionary game dynamic by combining the imitation and aspiration-driven update processes. The target was to clarify the influences of the aspiration-driven process on the evolution of the level of cooperation by considering the behavior of a population in which individuals have two strategies available: cooperation and defection. Through a numerical analysis of unstructured populations and simulation analyses of structured populations and of the random-matching model, the following results were found. First, the mean fraction of cooperators varied alongside the probability with which the individual adopted the aspiration-driven update rule. In the Prisoner's Dilemma and coexistence games, the aspiration-driven update process promoted cooperation in the well-mixed population but inhibited it in structured ones and the random-matching model; however, in the coordination game, the aspiration-driven update process was seen to exert the opposite effect on cooperation by inhibiting the latter in a homogeneously mixed population but promoting it in structured ones and in the random-matching model. Second, the mean fraction of cooperators changed with the aspiration level in the differently structured populations and random-matching model, and there appeared a phase transition point. Third, the evolutionary characteristics of the mean fraction of cooperators maintained robustness in the differently structured populations and random-matching model. These results extend evolutionary game theory.

Evolutionary multiplayer games on graphs with edge diversity

Evolutionary game dynamics in structured populations has been extensively explored in past decades. However, most previous studies assume that payoffs of individuals are fully determined by the strategic behaviors of interacting parties, and social ties between them only serve as the indicator of the existence of interactions. This assumption neglects important information carried by inter-personal social ties such as genetic similarity, geographic proximity, and social closeness, which may crucially affect the outcome of interactions. To model these situations, we present a framework of evolutionary multiplayer games on graphs with edge diversity, where different types of edges describe diverse social ties. Strategic behaviors together with social ties determine the resulting payoffs of interactants. Under weak selection, we provide a general formula to predict the success of one behavior over the other. We apply this formula to various examples which cannot be dealt with using previous models, including the division of labor and relationship- or edge-dependent games. We find that labor division can promote collective cooperation markedly. The evolutionary process based on relationship-dependent games can be approximated by interactions under a transformed and unified game. Our work stresses the importance of social ties and provides effective methods to reduce the calculating complexity in analyzing the evolution of realistic systems.

Individualised aspiration dynamics: Calculation by proofs

Cooperation is key for the evolution of biological systems ranging from bacteria communities to human societies. Evolutionary processes can dramatically alter the cooperation level. Evolutionary processes are typically of two classes: comparison based and self-evaluation based. The fate of cooperation is extremely sensitive to the details of comparison based processes. For self-evaluation processes, however, it is still unclear whether the sensitivity remains. We concentrate on a class of self-evaluation processes based on aspiration, where all the individuals adjust behaviors based on their own aspirations. We prove that the evolutionary outcome with heterogeneous aspirations is the same as that of the homogeneous one for regular networks under weak selection limit. Simulation results further suggest that it is also valid for general networks across various distributions of personalised aspirations. Our result clearly indicates that self-evaluation processes are robust in contrast with comparison based rules. In addition, our result greatly simplifies the calculation of the aspiration dynamics, which is computationally expensive.

Cooperation is the cornerstone to understand how biological systems evolve. Previous studies have shown that cooperation is sensitive to the details of evolutionary processes, even if all the individuals update strategies in the same way. Here we propose a class of updating rules driven by self-evaluation, where each individual has its personal aspiration. The evolutionary outcome is the same as if all the individuals adopt the same aspiration for regular networks, provided the selection intensity is weak enough. In addition, we provide a simple numerical method to identify the favored strategy. Our result shows a very robust class of strategy updating rules. And it implies that complexity in updating rules does not necessarily lead to the sensitivity of evolutionary outcomes.

Promotion of cooperation in evolutionary game dynamics with local information

In this paper, we propose a strategy-updating rule driven by local information, which is called Local process. Unlike the standard Moran process, the Local process does not require global information about the strategic environment. By analyzing the dynamical behavior of the system, we explore how the local information influences the fixation of cooperation in two-player evolutionary games. Under weak selection, the decreasing local information leads to an increase of the fixation probability when natural selection does not favor cooperation replacing defection. In the limit of sufficiently large selection, the analytical results indicate that the fixation probability increases with the decrease of the local information, irrespective of the evolutionary games. Furthermore, for the dominance of defection games under weak selection and for coexistence games, the decreasing of local information will lead to a speedup of a single cooperator taking over the population. Overall, to some extent, the local information is conducive to promoting the cooperation.

Reinforcement learning account of network reciprocity

Evolutionary game theory predicts that cooperation in social dilemma games is promoted when agents are connected as a network. However, when networks are fixed over time, humans do not necessarily show enhanced mutual cooperation. Here we show that reinforcement learning (specifically, the so-called Bush-Mosteller model) approximately explains the experimentally observed network reciprocity and the lack thereof in a parameter region spanned by the benefit-to-cost ratio and the node’s degree. Thus, we significantly extend previously obtained numerical results.

Impact of migration on the multi-strategy selection in finite group-structured populations

For large quantities of spatial models, the multi-strategy selection under weak selection is the sum of two competition terms: the pairwise competition and the competition of multiple strategies with equal frequency. Two parameters σ₁ and σ₂ quantify the dependence of the multi-strategy selection on these two terms, respectively. Unlike previous studies, we here do not require large populations for calculating σ₁ and σ₂, and perform the first quantitative analysis of the effect of migration on them in group-structured populations of any finite sizes. The Moran and the Wright-Fisher process have the following common findings. Compared with well-mixed populations, migration causes σ₁ to change with the mutation probability from a decreasing curve to an inverted U-shaped curve and maintains the increase of σ₂. Migration (probability and range) leads to a significant change of σ₁ but a negligible one of σ₂. The way that migration changes σ₁ is qualitatively similar to its influence on the single parameter characterizing the two-strategy selection. The Moran process is more effective in increasing σ₁ for most migration probabilities and the Wright-Fisher process is always more effective in increasing σ₂. Finally, our findings are used to study the evolution of cooperation under direct reciprocity.

Aspiration promotes cooperation in the prisoner's dilemma game with the imitation rule

A model of stochastic evolutionary game dynamics with finite population of size N+M was built. Among these individuals, N individuals update strategies with aspiration updating, while the other M individuals update strategies with imitation updating. In the proposed model, we obtain the expression of the mean fraction of cooperators and analyze some concrete cases. Compared with the standard imitation dynamics, there is always a positive probability to support the formation of cooperation in the system with the aspiration and imitation rules. Moreover, the numerical results indicate that more aspiration-driven individuals lead to a higher mean fraction of imitation-driven cooperators, which means the invasion of the aspiration-driven individuals is conducive to promoting the cooperation of the imitation-driven individuals.