Stochastic win-stay-lose-shift strategy with dynamic aspirations in evolutionary social dilemmas

Heterogeneous contributions can jeopardize cooperation in the public goods game

When studying social dilemma games, a crucial question arises regarding the impact of general heterogeneity on cooperation, which has been shown to have positive effects in numerous studies. Here, we demonstrate that heterogeneity in the contribution value for the focal public goods game can jeopardize cooperation. We show that there is an optimal contribution value in the homogeneous case that most benefits cooperation depending on the lattice. In a heterogeneous scenario, where strategy and contribution coevolve, cooperators making contributions higher than the optimal value end up harming those who contribute less. This effect is notably detrimental to cooperation in the square lattice with von Neumann neighborhood, while it can have no impact in other lattices. Furthermore, in parameter regions where a higher-contributing cooperator cannot normally survive alone, the exploitation of lower-value contribution cooperators allows their survival, resembling a parasitic behavior. To obtain these results, we examined the effect of various distributions for the contribution values in the initial condition and we conducted Monte Carlo simulations.

Super-rational aspiration promotes cooperation in the asymmetric game with peer exit punishment and reward

Super-rational aspiration induced strategy updating with exit rights has been considered in some previous studies, in which the players adjust strategies in line with their payoffs and aspirations, and they have access to exit the game. However, exit payoffs for exiting players are automatically allocated, which is clearly contrary to reality. In this study, evolutionary cooperation dynamics with super-rational aspiration and asymmetry in the Prisoner's Dilemma game is investigated, where exit payoffs are implemented by local peers. The results show that for different population structures, the asymmetry of the system is always contributive to the participation of the players. Furthermore, we show that under different exit payoffs, super-rationality and asymmetry are conductive to the evolution of cooperation.

Deterministic and stochastic cooperation transitions in evolutionary games on networks

Although the cooperative dynamics emerging from a network of interacting players has been exhaustively investigated, it is not yet fully understood when and how network reciprocity drives cooperation transitions. In this work, we investigate the critical behavior of evolutionary social dilemmas on structured populations by using the framework of master equations and Monte Carlo simulations. The developed theory describes the existence of absorbing, quasiabsorbing, and mixed strategy states and the transition nature, continuous or discontinuous, between the states as the parameters of the system change. In particular, when the decision-making process is deterministic, in the limit of zero effective temperature of the Fermi function, we find that the copying probabilities are discontinuous functions of the system's parameters and of the network degrees sequence. This may induce abrupt changes in the final state for any system size, in excellent agreement with the Monte Carlo simulation results. Our analysis also reveals the existence of continuous and discontinuous phase transitions for large systems as the temperature increases, which is explained in the mean-field approximation. Interestingly, for some game parameters, we find optimal "social temperatures" maximizing or minimizing the cooperation frequency or density.

Incorporating social payoff into reinforcement learning promotes cooperation

Reinforcement learning has been demonstrated to be an effective approach to investigate the dynamic of strategy updating and the learning process of agents in game theory. Most studies have shown that Q-learning failed to resolve the problem of cooperation in well-mixed populations or homogeneous networks. To this aim, we investigate the self-regarding Q-learning's effect on cooperation in spatial prisoner's dilemma games by incorporating the social payoff. Here, we redefine the reward term of self-regarding Q-learning by involving the social payoff; that is, the reward is defined as a monotonic function of the individual payoff and the social payoff represented by its neighbors' payoff. Numerical simulations reveal that such a framework can facilitate cooperation remarkably because the social payoff ensures agents learn to cooperate toward socially optimal outcomes. Moreover, we find that self-regarding Q-learning is an innovative rule that ensures cooperators coexist with defectors even at high temptations to defection. The investigation of the emergence and stability of the sublattice-ordered structure shows that such a mechanism tends to generate a checkerboard pattern to increase agents' payoff. Finally, the effects of Q-learning parameters are also analyzed, and the robustness of this mechanism is verified on different networks.

Two-layer network model of public goods games with intervention and corruption

Public goods games are widely used to model social dilemmas involving multiple agents. Though defection is the only rational choice for an individual in a public goods game, cooperative behavior is observed in a variety of social dilemmas, which is the subject of our research. Punishing defectors has been shown to be an effective mechanism for promoting cooperation, but it relies on the third-party umpire being fair. In this article, an umpire intervention model with corruption is proposed to explore the impact of corruption on punishment mechanisms. In our model, players and umpires operate in a multilayer network. The players play public goods games, which are overseen by umpires. Fair umpires punish defectors, whereas corrupt umpires take bribes from defectors rather than meting out a punishment. We separately explore the effects of the fraction of fair umpires _ρ, the spatial distribution, and the fine cost α and bribe cost β. Our Monte Carlo simulation shows that the above factors have a significant impact on cooperation. Intervention by an umpire always improves social efficiency, even for an entirely corrupt system. Moreover, relatively developed systems can resist corruption. Staggered and centralized distributions always have opposite effects on cooperative behavior, and these effects depend on _ρ and r. We also find that whether cooperators fully occupy the player layer depends only on whether β reaches a certain threshold.

Impact of the baseline payoff on evolutionary outcomes

Do individuals enjoying a higher baseline payoff behave similarly in competitive scenarios compared to their counterparts? The classical replicator equation does not answer such a question since it is invariant to the background or baseline payoff of individuals. In reality, however, if one's baseline payoff is higher than the possible payoffs of an interaction (or game), the individual may respond generously or indifferently if s(he) is satisfied with the prevailing benchmark payoff. This work intends to explore such a phenomenon within the realm of pairwise interactions-taking the prisoner's dilemma as a metaphor-in well-mixed finite and infinite populations. In this framework, a player uses the payoff (comprising baseline and game payoffs) -expectation difference to estimate a degree of eagerness and, with that degree of eagerness, revises his or her strategy with a certain probability. We adopt two approaches to explore such a context, naming them as the Fermi and imitation processes, in which the former uses a pairwise Femi function and the latter considers the relative fitness to estimate probabilities for strategy revision. In a finite population, we examine the effect of intensities to payoff-expectation and strategic payoff differences (denoted by k_{1} and k_{2}, respectively) as well as the level of contentment (ω) on the fixation probability and fixation time (for a single defector). We observe that the fixation probability surges with the increase of intensity parameters. Nevertheless, the maximum fixation probability may require a substantially larger time to fixate, especially when the expectation is lower than the baseline payoff. This means that cooperators can persist for a longer period of time. A higher expectation or greed, however, considerably reduces the fixation time. Interestingly, our numerical simulation reveals that both approaches are equivalent under weak k_{2}(≪1) in the Fermi process. We further derive mean-field equations for both approaches in the context of an infinite population, where we observe two possible evolutionary consequences: either full-scale defection or the persistence of the initial frequency of cooperators. The latter scenario indicates players' uninterested or neutral behavior in relation to the interaction due to their satisfaction on the baseline payoff.

Critical phenomena and strategy ordering with hub centrality approach in the aspiration-based coordination game

We study the coordination game with an aspiration-driven update rule in regular graphs and scale-free networks. We prove that the model coincides exactly with the Ising model and shows a phase transition at the critical selection noise when the aspiration level is zero. It is found that the critical selection noise decreases with clustering in random regular graphs. With a non-zero aspiration level, the model also exhibits a phase transition as long as the aspiration level is smaller than the degree of graphs. We also show that the critical exponents are independent of clustering and aspiration level to confirm that the coordination game belongs to the Ising universality class. As for scale-free networks, the effect of aspiration level on the order parameter at a low selection noise is examined. In model networks (the Barabási-Albert network and the Holme-Kim network), the order parameter abruptly decreases when the aspiration level is the same as the average degree of the network. In contrast, in real-world networks, the order parameter decreases gradually. We explain this difference by proposing the concepts of hub centrality and local hub. The histogram of hub centrality of real-world networks separates into two parts unlike model networks, and local hubs exist only in real-world networks. We conclude that the difference of network structures in model and real-world networks induces qualitatively different behavior in the coordination game.

Imitation and aspiration dynamics bring different evolutionary outcomes in feedback-evolving games

Feedback-evolving games characterize the interplay between the evolution of strategies and environments. Rich dynamics have been derived for such games under the premise of the replicator equation, which unveils persistent oscillations between cooperation and defection. Besides replicator dynamics, here we have employed aspiration dynamics, in which individuals, instead of comparing payoffs with opposite strategies, assess their payoffs by self-evaluation to update strategies. We start with a brief review of feedback-evolving games with replicator dynamics and then comprehensively discuss such games with aspiration dynamics. Interestingly, the tenacious cycles, as perceived in replicator dynamics, cannot be observed in aspiration dynamics. Our analysis reveals that a parameter θ —which depicts the strength of cooperation in enhancing the environment—plays a pivotal role in comprehending the dynamics. In particular, with the symmetric aspiration level, if replete and depleted states, respectively, experience Prisoner's Dilemma and Trivial games, the rich environment is achievable only when θ  > 1. The case θ  < 1 never allows us to reach the replete state, even with a higher cooperation level. Furthermore, if cooperators aspire less than defectors, then the enhanced state can be achieved with a relatively lower θ value compared with the opposite scenario because too much expectation from cooperation can be less beneficial.

Dynamic aspiration based on Win-Stay-Lose-Learn rule in spatial prisoner's dilemma game

Prisoner’s dilemma game is the most commonly used model of spatial evolutionary game which is considered as a paradigm to portray competition among selfish individuals. In recent years, Win-Stay-Lose-Learn, a strategy updating rule base on aspiration, has been proved to be an effective model to promote cooperation in spatial prisoner’s dilemma game, which leads aspiration to receive lots of attention. In this paper, according to Expected Value Theory and Achievement Motivation Theory, we propose a dynamic aspiration model based on Win-Stay-Lose-Learn rule in which individual’s aspiration is inspired by its payoff. It is found that dynamic aspiration has a significant impact on the evolution process, and different initial aspirations lead to different results, which are called Stable Coexistence under Low Aspiration, Dependent Coexistence under Moderate aspiration and Defection Explosion under High Aspiration respectively. Furthermore, a deep analysis is performed on the local structures which cause defectors’ re-expansion, the concept of END- and EXP-periods are used to justify the mechanism of network reciprocity in view of time-evolution, typical feature nodes for defectors’ re-expansion called Infectors, Infected nodes and High-risk cooperators respectively are found. Compared to fixed aspiration model, dynamic aspiration introduces a more satisfactory explanation on population evolution laws and can promote deeper comprehension for the principle of prisoner’s dilemma.

Evolution of cooperation in social dilemmas under the coexistence of aspiration and imitation mechanisms

Imitation and aspiration update rules are frequently observed in human and animal populations. While the imitation process entails payoff comparisons with surroundings, the aspiration process refers to self-evaluation. This work explores the evolution of cooperation in dyadic games under the coexistence of these two dynamics in an infinitely large well-mixed population. Two situations have been explored: (i) individuals adopt either an imitation or aspiration update rule with a certain probability, and (ii) the entire population is divided into two groups where one group only uses imitative rules and the other obeys aspiration updating alone. Both premises have been modeled by taking an infinite approximation of the finite population. In particular, the second mixing principle follows an additive property: the outcome of the whole population is the weighted average of outcomes from imitators and aspiration-driven individuals. Our work progressively investigates several variants of aspiration dynamics under strong selection, encompassing symmetric, asymmetric, and adaptive aspirations, which then coalesce with imitative dynamics. We also demonstrate which of the update rules performs better, under different social dilemmas, by allowing the evolution of the preference of update rules besides strategies. Aspiration dynamics always outperform imitation dynamics in the prisoner's dilemma, however, in the chicken and stag-hunt games the predominance of either update rule depends on the level of aspirations as well as on the extent of greed and fear present in the system. Finally, we examine the coevolution of strategies, aspirations, and update rules which leads to a binary state of obeying either imitation or aspiration dynamics. In such a circumstance, when aspiration dynamics prevail over imitation dynamics, cooperators and defectors coexist to an equal extent.

Vaccinating behaviour guided by imitation and aspiration

Vaccinating decisions can be influenced by imitation as well as self-evaluation or aspiration. This work analyses vaccinating behaviours by coupling both imitation and aspiration update rules, adopting an existing set-up of the mean-field vaccination game. We incorporate the imitation mechanism with several variants of the aspiration protocol, encompassing constant and adaptive aspirations. Equations of the combined dynamics have been derived by grouping the population according to (i) vaccinating strategies and (ii) healthy and infected status within each strategy. If aspiration levels are fixed but differentiated by vaccinating strategies, then vaccinators aspiring less than non-vaccinators are found to ameliorate the vaccination coverage, thereby yielding a less infectious state. The adaptive aspirations maintain a positive correlation with the vaccine efficacy while keeping the opposite relation with vaccination cost. When vaccinating strategies, aspirations and update rules are allowed to evolve synchronously, then either the imitation or aspiration process takes over the entire population. If aspiration rules prevail, then vaccinees and non-vaccinees coexist equally (according to (i)) or vaccine uptake follows a non-monotonic trend with the efficacy (according to (ii)). The imitative rule performs better when vaccination is less expensive or cheap, while aspiration updating safeguards the tenacity of vaccinees despite vaccination being expensive.

Strategy equilibrium in dilemma games with off-diagonal payoff perturbations

We analyze the strategy equilibrium of dilemma games considering a payoff matrix affected by small and random perturbations on the off-diagonal. Notably, a recent work [Proc. R. Soc. A 476, 20200116 (2020)1364-502110.1098/rspa.2020.0116] reported that while cooperation is sustained by perturbations acting on the main diagonal, a less clear scenario emerges when perturbations act on the off-diagonal. Thus, the second case represents the core of this investigation, aimed at completing the description of the effects that payoff perturbations have on the dynamics of evolutionary games. Our results, achieved by analyzing the proposed model under a variety of configurations as different update rules, suggest that off-diagonal perturbations actually constitute a nontrivial form of noise. In particular, the most interesting effects are detected near the phase transition, as perturbations tend to move the strategy distribution towards nonordered states of equilibrium, supporting cooperation when defection is pervading the population, and supporting defection in the opposite case. To conclude, we identified a form of noise that, under controlled conditions, could be used to enhance cooperation and greatly delay its extinction.

Direct reciprocity and model-predictive rationality explain network reciprocity over social ties

Since M. A. Nowak & R. May’s (1992) influential paper, limiting each agent’s interactions to a few neighbors in a network of contacts has been proposed as the simplest mechanism to support the evolution of cooperation in biological and socio-economic systems. The network allows cooperative agents to self-assort into clusters, within which they reciprocate cooperation. This (induced) network reciprocity has been observed in several theoreticalmodels and shown to predict the fixation of cooperation under a simple rule: the benefit produced by an act of cooperation must outweigh the cost of cooperating with all neighbors. However, the experimental evidence among humans is controversial: though the rule seems to be confirmed, the underlying modeling assumptions are not. Specifically, models assume that agents update their strategies by imitating better performing neighbors, even though imitation lacks rationality when interactions are far from all-to-all. Indeed, imitation did not emerge in experiments. What did emerge is that humans are conditioned by their own mood and that, when in a cooperative mood, they reciprocate cooperation. To help resolve the controversy, we design a model in which we rationally confront the two main behaviors emerging from experiments—reciprocal cooperation and unconditional defection—in a networked prisoner’s dilemma. Rationality is introduced by means of a predictive rule for strategy update and is bounded by the assumed model society. We show that both reciprocity and a multi-step predictive horizon are necessary to stabilize cooperation, and sufficient for its fixation, provided the game benefit-to-cost ratio is larger than a measure of network connectivity. We hence rediscover the rule of network reciprocity, underpinned however by a different evolutionary mechanism.

Heterogeneous update mechanisms in evolutionary games: Mixing innovative and imitative dynamics

Innovation and evolution are two processes of paramount relevance for social and biological systems. In general, the former allows the introduction of elements of novelty, while the latter is responsible for the motion of a system in its phase space. Often, these processes are strongly related, since an innovation can trigger the evolution, and the latter can provide the optimal conditions for the emergence of innovations. Both processes can be studied by using the framework of evolutionary game theory, where evolution constitutes an intrinsic mechanism. At the same time, the concept of innovation requires an opportune mathematical representation. Notably, innovation can be modeled as a strategy, or it can constitute the underlying mechanism that allows agents to change strategy. Here, we analyze the second case, investigating the behavior of a heterogeneous population, composed of imitative and innovative agents. Imitative agents change strategy only by imitating that of their neighbors, whereas innovative ones change strategy without the need for a copying source. The proposed model is analyzed by means of analytical calculations and numerical simulations in different topologies. Remarkably, results indicate that the mixing of mechanisms can be detrimental to cooperation near phase transitions. In those regions, the spatial reciprocity from imitative mechanisms is destroyed by innovative agents, leading to the downfall of cooperation. Our investigation sheds some light on the complex dynamics emerging from the heterogeneity of strategy revision methods, highlighting the role of innovation in evolutionary games.

Coevolution of teaching ability and cooperation in spatial evolutionary games

Individuals with higher reputation are able to spread their social strategies easily. At the same time, one’s reputation is changing according to his previous behaviors, which leads to completely different teaching abilities for players. To explore the effect of the teaching ability influenced by reputation, we consider a coevolutionary model in which the reputation score affects the updating rule in spatial evolutionary games. More precisely, the updating probability becomes bigger if his/her partner has a positive reputation. Otherwise, the updating probability becomes smaller. This simple design describes the influence of teaching ability on strategy adoption effectively. Numerical results focus on the proportion of cooperation under different levels of the amplitude of change of reputation and the range of reputation. For this dynamics, the fraction of cooperators presents a growth trend within a wide range of parameters. In addition, to validate the generality of this mechanism, we also employ the snowdrift game. Moreover, the evolution of cooperation on Erdős-Rényi random graph is studied for the prisoner’s dilemma game. Our results may be conducive to understanding the emergence and sustainability of cooperation during the strategy adoptions in reality.

Phase diagram and criticality of the two-dimensional prisoner's dilemma model

The stationary states of the prisoner's dilemma model are studied on a square lattice taking into account the role of a noise parameter in the decision-making process. Only first neighboring players-defectors and cooperators-are considered in each step of the game. Through Monte Carlo simulations we determined the phase diagrams of the model in the plane noise versus the temptation to defect for a large range of values of the noise parameter. We observed three phases: cooperators and defectors absorbing phases, and a coexistence phase between them. The phase transitions as well as the critical exponents associated with them were determined using both static and dynamical scaling laws.

Social Dilemma Analysis of the Spread of Infectious Disease

Understanding and controlling the spread of infectious disease is a pressing issue for our society. Contemporary globally connected civilization is more at risk from various modern infectious diseases than classical ones such as pests, cholera, and tuberculosis. Over the last few years, pandemic outbreaks of highly virulent influenza, possibly related to avian flu, severe acute respiratory syndrome (SARS), and middle-eastern respiratory syndrome coronavirus (MARSE) have been a threat. Beyond this, the intentional spread of infectious disease, e.g., “bioterrorism”, has come to be recognized as being just as dangerous as nuclear weapons. An infectious disease spreads on human social networks. Each individual can protect himself through several measures. Pre-emptive vaccination is thought to be most effective, although it incurs a partial cost to each individual. This brings about a social dilemma, because an individual may be able to rely on so-called “herd immunity” to avoid his own infection without himself being vaccinated. Also, besides vaccination, there may be several practical ways to protect against contagion, such as wearing a mask, keeping away from crowds, and self-isolation by leaving the home less often, which may be less costly and less effective than vaccination. In any case, there is a human-decision-making process regarding what steps should be taken, while the dynamics of infectious-disease spread can themselves be evaluated as a diffusion problem that has been well-studied in physics for many years. Thus, based on the concept of human–environment–social interaction, a basic-physics model for this diffusion problem that considers evolutionary game theory (EGT) may lead us to obtain some meaningful solutions that can be proposed to our society. Following the previous chapter explaining how EGT can be applied to traffic-flow analysis, this chapter describes this practical problem.

The Beneficial Role of Mobility for the Emergence of Innovation

Innovation is a key ingredient for the evolution of several systems, including social and biological ones. Focused investigations and lateral thinking may lead to innovation, as well as serendipity and other random discovery processes. Some individuals are talented at proposing innovation (say innovators), while others at deeply exploring proposed novelties, at getting further insights on a theory, or at developing products, services, and so on (say developers). This separation in terms of innovators and developers raises an issue of paramount importance: under which conditions a system is able to maintain innovators? According to a simple model, this work investigates the evolutionary dynamics that characterize the emergence of innovation. In particular, we consider a population of innovators and developers, in which agents form small groups whose composition is crucial for their payoff. The latter depends on the heterogeneity of the formed groups, on the amount of innovators they include, and on an award-factor that represents the policy of the system for promoting innovation. Under the hypothesis that a "mobility" effect may support the emergence of innovation, we compare the equilibria reached by our population in different cases. Results confirm the beneficial role of "mobility", and the emergence of further interesting phenomena.

The co-evolution of networks and prisoner's dilemma game by considering sensitivity and visibility

Strategies adopted by individuals in a social network significantly impact the network, and they strongly affect relationships between individuals in the network. Links between individuals also heavily influence their levels of cooperation. Taking into account the evolution of each individual's connection, we explore how sensitivity and visibility affect the prisoner's dilemma game. The so-called 'sensitivity' and 'visibility' respectively present one's self-protection consciousness and the ability of gaining information. We find that at moderate levels of player sensitivity cooperative behavior increases, but that at high levels it is inhibited. We also find that the heterogeneity of the weight of individuals at the end of the game is higher when sensitivity and visibility are increased, but that the successful-defection-payoff has less impact on the weight of individuals and on the relationship between the heterogeneity of the weight of individuals and the density of cooperators. This framework can be used to clarify the interaction mechanism between the micro-level of individual behavior and the macro-level of individual co-evolutionary processes.

Win-stay-lose-learn promotes cooperation in the prisoner's dilemma game with voluntary participation

Voluntary participation, demonstrated to be a simple yet effective mechanism to promote persistent cooperative behavior, has been extensively studied. It has also been verified that the aspiration-based win-stay-lose-learn strategy updating rule promotes the evolution of cooperation. Inspired by this well-known fact, we combine the Win-Stay-Lose-Learn updating rule with voluntary participation: Players maintain their strategies when they are satisfied, or players attempt to imitate the strategy of one randomly chosen neighbor. We find that this mechanism maintains persistent cooperative behavior, even further promotes the evolution of cooperation under certain conditions.

From degree-correlated to payoff-correlated activity for an optimal resolution of social dilemmas

An active participation of players in evolutionary games depends on several factors, ranging from personal stakes to the properties of the interaction network. Diverse activity patterns thus have to be taken into account when studying the evolution of cooperation in social dilemmas. Here we study the weak prisoner's dilemma game, where the activity of each player is determined in a probabilistic manner either by its degree or by its payoff. While degree-correlated activity introduces cascading failures of cooperation that are particularly severe on scale-free networks with frequently inactive hubs, payoff-correlated activity provides a more nuanced activity profile, which ultimately hinders systemic breakdowns of cooperation. To determine optimal conditions for the evolution of cooperation, we introduce an exponential decay to payoff-correlated activity that determines how fast the activity of a player returns to its default state. We show that there exists an intermediate decay rate at which the resolution of the social dilemma is optimal. This can be explained by the emerging activity patterns of players, where the inactivity of hubs is compensated effectively by the increased activity of average-degree players, who through their collective influence in the network sustain a higher level of cooperation. The sudden drops in the fraction of cooperators observed with degree-correlated activity therefore vanish, and so does the need for the lengthy spatiotemporal reorganization of compact cooperative clusters. The absence of such asymmetric dynamic instabilities thus leads to an optimal resolution of social dilemmas, especially when the conditions for the evolution of cooperation are strongly adverse.