Voluntary rewards mediate the evolution of pool punishment for maintaining public goods in large populations

Emergence of cooperation under punishment: A reinforcement learning perspective

Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning framework where players imitate the strategies of those who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their experience and long-term returns. Specifically, we investigate the prisoners' dilemma game with a Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Unexpectedly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid-gas transition. The analysis of a Q-table reveals the evolution of the underlying "psychologic" changes, which explains the nucleation process and different levels of cooperation. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.

Evolutionary dynamics of any multiplayer game on regular graphs

Multiplayer games on graphs are at the heart of theoretical descriptions of key evolutionary processes that govern vital social and natural systems. However, a comprehensive theoretical framework for solving multiplayer games with an arbitrary number of strategies on graphs is still missing. Here, we solve this by drawing an analogy with the Balls-and-Boxes problem, based on which we show that the local configuration of multiplayer games on graphs is equivalent to distributing k identical co-players among n distinct strategies. We use this to derive the replicator equation for any n-strategy multiplayer game under weak selection, which can be solved in polynomial time. As an example, we revisit the second-order free-riding problem, where costly punishment cannot truly resolve social dilemmas in a well-mixed population. Yet, in structured populations, we derive an accurate threshold for the punishment strength, beyond which punishment can either lead to the extinction of defection or transform the system into a rock-paper-scissors-like cycle. The analytical solution also qualitatively agrees with the phase diagrams that were previously obtained for non-marginal selection strengths. Our framework thus allows an exploration of any multi-strategy multiplayer game on regular graphs.

The evolution of cooperation in the unidirectional division of labour on a tree network

Division of labour on complex networks is rarely investigated using evolutionary game theory. We investigate a division of labour where divided roles are assigned to groups on the nodes of a general unidirectional finite tree graph network. From the network's original node, a task flows and is divided along the branches. A player is randomly selected in each group of cooperators and defectors, who receives a benefit from a cooperator in the upstream group and a part of the task. A cooperator completes their part by paying a cost and then passing it downstream until the entire task is completed. Defectors do not do anything and the division of labour stops, causing all groups to suffer losses due to the incomplete task. We develop a novel method to analyse the local stability in this general tree. We discover that not the benefits but the costs of the cooperation influence the evolution of cooperation, and defections in groups that are directly related to that group's task cause damage to players in that group. We introduce two sanction systems, one of which induces the evolution of cooperation more than the system without sanctions, and promote the coexistence of cooperator and defector groups.

Promoting Low-Carbon Purchase from Social Norms Perspective

The importance of individual consumption behavior in a low-carbon economy is gradually recognized. Social norms have a significant effect on individual purchase behavior. However, the influence mechanism of social norms still needs more research. We conducted two behavioral experiments to explore the specific factors: first, the effect of descriptive norms on personal low-carbon consumption behavior through feedback information, and second, a comparison with injunctive norms, focusing on the impact of the normative focus shift brought by punishment represented by the policy implementation. The results show that social norms can effectively promote individual low-carbon consumption through feedback and high policy implementation efficiency. In particular, after effective policy implementation becomes an inherent element of injunctive norms, injunctive norms are activated and become the focus of norms, significantly improving the purchase rate of low-carbon goods.

The evolution of cooperation in the unidirectional linear division of labour of finite roles

Evolution of cooperation is a puzzle in evolutionary biology and social sciences. Previous studies assumed that players are equal and have symmetric relationships. In our society, players are in different roles, have an asymmetric relationship and cooperate together. We focused on the linear division of labour in a unidirectional chain that has finite roles, each of which is assigned to one group with cooperators and defectors. A cooperator in an upstream group produces and modifies a product, paying a cost of cooperation, and hands it to a player in a downstream group who obtains the benefit from the product. If players in all roles cooperate, a final product can be completed. However, if a player in a group chooses defection, the division of labour stops, the final product cannot be completed and all players in all roles suffer damage. By using the replicator equations of the asymmetric game, we investigate which sanction system promotes the evolution of cooperation in the division of labour. We find that not the benefit of the product but the cost of cooperation matters to the evolutionary dynamics and that the probability of finding a defector determines which sanction system promotes the evolution of cooperation.

The coordination of collective and individual solutions in risk-resistant scenarios

ABSTRACT

Human societies are constantly coping with global risks. In the face of these risks, people typically have two options, that is, to respond together as a whole (collective solution) or to respond independently (individual solution). Based on these two solutions, individuals have a variety of behavioral strategies. On the other hand, various regulatory bodies supported by the population limit people's choices and punish individuals who do not contribute to collective solutions. So with different risks, how do the two solutions, the various individual strategies, and the constraints from regulators affect the group's response to risk? This paper proposes an extended public goods game model involving opportunists and the regulator to explore the effectiveness of collective and individual solutions against risks. The results show that requiring individuals to invest more in the collective solution reduces the group' s success in resisting risk. To improve the group's ability to resist risk, investment in individual solution should be at least no less than that in collective solution. The establishment fund and punishment intensity of the regulatory agency have no significant effect on the success of collective and individual solutions. This inspires us to contemplate the role and measures of various types of authorities in coping with global risks.

Authority or Autonomy? Exploring Interactions between Central and Peer Punishments in Risk-Resistant Scenarios

Game theory provides a powerful means to study human cooperation and better understand cooperation-facilitating mechanisms in general. In classical game-theoretic models, an increase in group cooperation constantly increases people's gains, implying that individual gains are a continuously varying function of the cooperation rate. However, this is inconsistent with the increasing number of risk-resistant scenarios in reality. A risk-resistant scenario means once a group does not successfully resist the risk, all individuals lose their resources, such as a community coping with COVID-19 and a village resisting a flood. In other words, individuals' gains are segmented about the collaboration rate. This paper builds a risk-resistant model to explore whether punishment still promotes collaboration when people resist risk. The results show that central and peer punishments can both encourage collaboration but with different characteristics under different risk-resistant scenarios. Specifically, central punishment constrains the collaboration motivated by peer punishment regardless of risk, while peer punishment limits the collaboration induced by central punishment only when the risk is high. Our findings provide insights into the balance between peer punishment from public autonomy and central punishment from central governance, and the proposed model paves the way for the development of richer risk-resistant models.

Evolution of prosocial punishment in unstructured and structured populations and in the presence of antisocial punishment

A large body of empirical evidence suggests that altruistic punishment abounds in human societies. Based on such evidence, it is suggested that punishment serves an important role in promoting cooperation in humans and possibly other species. However, as punishment is costly, its evolution is subject to the same problem that it tries to address. To suppress this so-called second-order free-rider problem, known theoretical models on the evolution of punishment resort to one of the few established mechanisms for the evolution of cooperation. This leaves the question of whether altruistic punishment can evolve and give rise to the evolution of cooperation in the absence of such auxiliary cooperation-favoring mechanisms unaddressed. Here, by considering a population of individuals who play a public goods game, followed by a public punishing game, introduced here, we show that altruistic punishment indeed evolves and promotes cooperation in the absence of a cooperation-favoring mechanism. In our model, the punishment pool is considered a public resource whose resources are used for punishment. We show that the evolution of a punishing institution is facilitated when resources in the punishment pool, instead of being wasted, are used to reward punishers when there is nobody to punish. Besides, we show that higher returns to the public resource or punishment pool facilitate the evolution of prosocial instead of antisocial punishment. We also show that an optimal cost of investment in the punishment pool facilitates the evolution of prosocial punishment. Finally, our analysis shows that being close to a physical phase transition facilitates the evolution of altruistic punishment.

Two ways to overcome the three social dilemmas of indirect reciprocity

Indirect reciprocity is one of the main principles of evolving cooperation in a social dilemma situation. In reciprocity, a positive score is given to cooperative behaviour while a negative score is given to non-cooperative behaviour, and the dilemma is resolved by selectively cooperating only with those with positive scores. However, many studies have shown that non-cooperation with those who have not cooperated also downgrades one's reputation; they have called this situation the scoring dilemma. To address this dilemma, the notion of justified punishments has been considered. The notion of justified punishment allows good individuals who defect against bad co-players to keep their standing. Despite numerous studies on justified punishment, it is unknown whether this solution leads to a new type of dilemma because reputations may be downgraded when the intent of punishment is not correctly communicated. The dilemma of punishment has so far been rarely analysed, and thus, the complete solution of the mechanism for evolving cooperation using the principle of indirect reciprocity has not been found yet. Here, we identify sufficient conditions to overcome each of the three dilemmas including the dilemma of punishment to maintain stable cooperation by using the framework of evolutionary game theory. This condition includes the principle of detecting free riders, which resolves the social dilemma, the principle of justification, which resolves the scoring dilemma, and the principle of generosity, which resolves the dilemma of punishment. A norm that satisfies these principles can stably maintain social cooperation. Our insights may offer a general assessment principle that applies to a wide range of subjects, from individual actions to national decisions.

The Singaporean model in public goods dilemmas with benevolent leaders and bribery

Public goods dilemmas are at the heart of some of the greatest challenges of our time, including climate inaction, growing inequality, and the overuse of natural resources. The public goods game in which cooperators contribute to a common pool that is then shared equally with defectors who contribute nothing captures the gist of the problem. Cooperators therefore cannot prevail, which ultimately leads to the tragedy of the commons. Actions such as punishment, rewards, and exclusion have been shown to help, but they are costly, therefore rendering cooperators second-order free-riders due to their lack of participation in these actions. In the search for a remedy, we study the public goods game with benevolent leaders who, at a personal cost, have the ability to exclude defectors from using common pool resources. We also consider bribers who can pay the leaders to relax their exclusion efforts. In a traditional setting, this setup yields the standard second-order free-rider problem, where, ironically, the leaders are overcome by cooperators, who then themselves succumb to defectors. We show, however, that the Singaporean model - where a leader's payoff is determined not only by the regular sharing income from the firm production but also by the success of gross firm production as an incentive - can resolve the second-order free-rider problem. We also show that the detrimental effect of bribery can always be, no matter how high the bribe, held in check as long as the number of individuals engaged in this activity is low compared to the number of benevolent leaders. Otherwise, an abrupt transition to a cooperator-less state becomes unavoidable. We discuss the implications of our research for designing successful cooperation and anti-corruption strategies in public goods dilemmas.

A Review of Theoretical Studies on Indirect Reciprocity

Despite the accumulation of research on indirect reciprocity over the past 30 years and the publication of over 100,000 related papers, there are still many issues to be addressed. Here, we look back on the research that has been done on indirect reciprocity and identify the issues that have been resolved and the ones that remain to be resolved. This manuscript introduces indirect reciprocity in the context of the evolution of cooperation, basic models of social dilemma situations, the path taken in the elaboration of mathematical analysis using evolutionary game theory, the discovery of image scoring norms, and the breakthroughs brought about by the analysis of the evolutionary instability of the norms. Moreover, it presents key results obtained by refining the assessment function, resolving the punishment dilemma, and presenting a complete solution to the social dilemma problem. Finally, it discusses the application of indirect reciprocity in various disciplines.

The competitive advantage of institutional reward

Sustaining cooperation among unrelated individuals is a fundamental challenge in biology and the social sciences. In human society, this problem can be solved by establishing incentive institutions that reward cooperators and punish free-riders. Most of the previous studies have focused on which incentives promote cooperation best. However, a higher cooperation level does not always imply higher group fitness, and only incentives that lead to higher fitness can survive in social evolution. In this paper, we compare the efficiencies of three types of institutional incentives, namely, reward, punishment, and a mixture of reward and punishment, by analysing the group fitness at the stable equilibria of evolutionary dynamics. We find that the optimal institutional incentive is sensitive to decision errors. When there is no error, a mixture of reward and punishment can lead to high levels of cooperation and fitness. However, for intermediate and large errors, reward performs best, and one should avoid punishment. The failure of punishment is caused by two reasons. First, punishment cannot maintain a high cooperation level. Second, punishing defectors almost always reduces the group fitness. Our findings highlight the role of reward in human cooperation. In an uncertain world, the institutional reward is not only effective but also efficient.

Co-Evolution of Complex Network Public Goods Game under the Edges Rules

The reconnection of broken edges is an effective way to avoid drawback for the commons in past studies. Inspired by this, we proposed a public goods game model under the edges rules, where we evaluate the weight of edges by their nodes' payoff. The results proved that the game obtains a larger range of cooperation with a small gain factor by this proposed model by consulting Monte Carlo simulations (MCS) and real experiments. Furthermore, as the following the course of game and discussing the reason of cooperation, in the research, we found that the distribution entropy of the excess average degree is able to embody and predict the presence of cooperation.

Probabilistic punishment and reward under rule of trust-based decision-making in continuous public goods game

Altruistic punishment and reward have been proved to promote the evolution of cooperation in the public goods game(PGG), but the punishers and the rewarders have to pay a price for these behaviors and that results in overall loss of interest. In present work, probabilistic punishment and reward are introduced to PGG. Probabilistic punishment and reward mean that punishment and reward are executed with a certain probability. Although that will reduce unnecessary costs, occasional absence of execution can lead to distrust. Thus we focus on how to implement punishment and reward efficiently within the structured populations. Numerical simulations are performed and prove that probabilistic punishment and reward can promote the evolution of cooperation more effectively. Further researches indicate that there is an optimal executing probability to promote cooperation and maximize reduction of cost. In addition, when the unit cost is high, the PGG with probabilistic punishment and reward still helps the evolution of altruistic punishers and rewarders, thereby avoiding collapse of cooperation.

Synergistic third-party rewarding and punishment in the public goods game

We study the evolution of cooperation in the spatial public goods game in the presence of third-party rewarding and punishment. The third party executes public intervention, punishing groups where cooperation is weak and rewarding groups where cooperation is strong. We consider four different scenarios to determine what works best for cooperation, in particular, neither rewarding nor punishment, only rewarding, only punishment or both rewarding and punishment. We observe strong synergistic effects when rewarding and punishment are simultaneously applied, which are absent if neither of the two incentives or just each individual incentive is applied by the third party. We find that public cooperation can be sustained at comparatively low third-party costs under adverse conditions, which is impossible if just positive or negative incentives are applied. We also examine the impact of defection tolerance and application frequency, showing that the higher the tolerance and the frequency of rewarding and punishment, the more cooperation thrives. Phase diagrams and characteristic spatial distributions of strategies are presented to corroborate these results, which will hopefully prove useful for more efficient public policies in support of cooperation in social dilemmas.

Benefits of asynchronous exclusion for the evolution of cooperation in stochastic evolutionary optional public goods games

Mechanisms and conditions for the spontaneous emergence of cooperation in multi-player social dilemma games remain an open question. This paper focuses on stochastic evolutionary optional public goods games with different exclusion strategies. We introduce four strategy types in the population, namely, cooperation, defection, loner and exclusion. Synchronous and asynchronous exclusion forms have been compared in finite-sized, well-mixed and structured populations. In addition, we verify that the asynchronous exclusion mechanism is indeed better than the synchronous exclusion mechanism in all cases. The benefits of the asynchronous exclusion are measured by comparing the probability that the system chooses the cooperative states in the two situations. In the well-mixed population cases, only when the investment amplification factor is small and the probability of exclusion success is high will the asynchronous exclusion mechanism have a relatively large advantage in promoting cooperation. However, in the structured population cases, the range of the investment amplification factor, in which the asynchronous exclusion mechanism has relatively large advantages in promoting cooperation, is somewhat different and is mainly in the middle of the interval under our parameters. Our study further corroborated that when non-participation and exclusion strategies exist, a structured population does not necessarily promote cooperation compared with a well-mixed population for some parameter combinations. Thus, we acquire a good understanding of the emergence of cooperation under different exclusion mechanisms.

Evolutionary dynamics in the public goods games with switching between punishment and exclusion

Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely, peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors, and players adopting the switching strategy in a well-mixed population, and this regardless of whether the pool-based or the peer-based switching strategy is introduced. Moreover, we show that the pure exclusion strategy alone is able to evoke a limit cycle attractor in the evolutionary dynamics, such that cooperation can coexist with other strategies.

Stochastic evolutionary voluntary public goods game with punishment in a Quasi-birth-and-death process

Traditional replication dynamic model and the corresponding concept of evolutionary stable strategy (ESS) only takes into account whether the system can return to the equilibrium after being subjected to a small disturbance. In the real world, due to continuous noise, the ESS of the system may not be stochastically stable. In this paper, a model of voluntary public goods game with punishment is studied in a stochastic situation. Unlike the existing model, we describe the evolutionary process of strategies in the population as a generalized quasi-birth-and-death process. And we investigate the stochastic stable equilibrium (SSE) instead. By numerical experiments, we get all possible SSEs of the system for any combination of parameters, and investigate the influence of parameters on the probabilities of the system to select different equilibriums. It is found that in the stochastic situation, the introduction of the punishment and non-participation strategies can change the evolutionary dynamics of the system and equilibrium of the game. There is a large range of parameters that the system selects the cooperative states as its SSE with a high probability. This result provides us an insight and control method for the evolution of cooperation in the public goods game in stochastic situations.

Antisocial rewarding in structured populations

Cooperation in collective action dilemmas usually breaks down in the absence of additional incentive mechanisms. This tragedy can be escaped if cooperators have the possibility to invest in reward funds that are shared exclusively among cooperators (prosocial rewarding). Yet, the presence of defectors who do not contribute to the public good but do reward themselves (antisocial rewarding) deters cooperation in the absence of additional countermeasures. A recent simulation study suggests that spatial structure is sufficient to prevent antisocial rewarding from deterring cooperation. Here we reinvestigate this issue assuming mixed strategies and weak selection on a game-theoretic model of social interactions, which we also validate using individual-based simulations. We show that increasing reward funds facilitates the maintenance of prosocial rewarding but prevents its invasion, and that spatial structure can sometimes select against the evolution of prosocial rewarding. Our results suggest that, even in spatially structured populations, additional mechanisms are required to prevent antisocial rewarding from deterring cooperation in public goods dilemmas.

Competitions between prosocial exclusions and punishments in finite populations

Prosocial punishment has been proved to be a powerful mean to promote cooperation. Recent studies have found that social exclusion, which indeed can be regarded as a kind of punishment, can also support cooperation. However, if prosocial punishment and exclusion are both present, it is still unclear which strategy is more advantageous to curb free-riders. Here we first study the direct competition between different types of punishment and exclusion. We find that pool (peer) exclusion can always outperform pool (peer) punishment both in the optional and in the compulsory public goods game, no matter whether second-order sanctioning is considered or not. Furthermore, peer exclusion does better than pool exclusion both in the optional and in the compulsory game, but the situation is reversed in the presence of second-order exclusion. Finally, we extend the competition among all possible sanctioning strategies and find that peer exclusion can outperform all other strategies in the absence of second-order exclusion and punishment, while pool exclusion prevails when second-order sanctioning is possible. Our results demonstrate that exclusion is a more powerful strategy than punishment for the resolution of social dilemmas.

The dynamics of human behavior in the public goods game with institutional incentives

The empirical research on the public goods game (PGG) indicates that both institutional rewards and institutional punishment can curb free-riding and that the punishment effect is stronger than the reward effect. Self-regarding models that are based on Nash equilibrium (NE) strategies or evolutionary game dynamics correctly predict which incentives are best at promoting cooperation, but individuals do not play these rational strategies overall. The goal of our study is to investigate the dynamics of human decision making in the repeated PGG with institutional incentives. We consider that an individual's contribution is affected by four factors, which are self-interest, the behavior of others, the reaction to rewards, and the reaction to punishment. We find that people on average do not react to rewards and punishment, and that self-interest and the behavior of others sufficiently explain the dynamics of human behavior. Further analysis suggests that institutional incentives promote cooperation by affecting the self-regarding preference and that the other-regarding preference seems to be independent of incentive schemes. Because individuals do not change their behavioral patterns even if they were not rewarded or punished, the mere potential to punish defectors and reward cooperators can lead to considerable increases in the level of cooperation.