1
|
Zhao C, Zhu Y. Heterogeneous decision-making dynamics of threshold-switching agents on complex networks. CHAOS (WOODBURY, N.Y.) 2023; 33:123133. [PMID: 38149990 DOI: 10.1063/5.0172442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 11/21/2023] [Indexed: 12/28/2023]
Abstract
In the classical two-player decision-making scenario, individuals may have different tendencies to take a certain action, given that there exists a sufficient number of neighbors adopting a particular option. This is ubiquitous in many real-life contexts including traffic congestion, crowd evacuation, and minimal vertex cover problem. Under best-response dynamics, we investigate the decision-making behaviors of heterogeneous agents on complex networks. Results of the networked games are twofold: for networks of uniform degree distribution (e.g., the lattice) and fraction of the strategy is of a linear function of the threshold setting. Moreover, the equilibrium analysis is provided and the relationship between the equilibrium dynamics and the change of the threshold value is given quantitatively. Next, if the games are played on networks with non-uniform degree distribution (e.g., random regular and scale-free networks), influence of the threshold-switching will be weakened. Robust experiments indicate that it is not the value of the average degree, but the degree distribution that influences how the strategy evolves affected by the threshold settings. Our result shows that the decision-making behaviors can be effectively manipulated by tuning the parameters in the utility function (i.e., thresholds) of some agents for more regular network structures.
Collapse
Affiliation(s)
- Chengli Zhao
- College of Liberal Arts and Sciences, National University of Defense Technology, Changsha 410073, China
| | - Yuying Zhu
- School of Artificial Intelligence, Tiangong University, Tianjin 300387, China
| |
Collapse
|
2
|
Fan L, Song Z, Wang L, Liu Y, Wang Z. Incorporating social payoff into reinforcement learning promotes cooperation. CHAOS (WOODBURY, N.Y.) 2022; 32:123140. [PMID: 36587319 DOI: 10.1063/5.0093996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 12/01/2022] [Indexed: 06/17/2023]
Abstract
Reinforcement learning has been demonstrated to be an effective approach to investigate the dynamic of strategy updating and the learning process of agents in game theory. Most studies have shown that Q-learning failed to resolve the problem of cooperation in well-mixed populations or homogeneous networks. To this aim, we investigate the self-regarding Q-learning's effect on cooperation in spatial prisoner's dilemma games by incorporating the social payoff. Here, we redefine the reward term of self-regarding Q-learning by involving the social payoff; that is, the reward is defined as a monotonic function of the individual payoff and the social payoff represented by its neighbors' payoff. Numerical simulations reveal that such a framework can facilitate cooperation remarkably because the social payoff ensures agents learn to cooperate toward socially optimal outcomes. Moreover, we find that self-regarding Q-learning is an innovative rule that ensures cooperators coexist with defectors even at high temptations to defection. The investigation of the emergence and stability of the sublattice-ordered structure shows that such a mechanism tends to generate a checkerboard pattern to increase agents' payoff. Finally, the effects of Q-learning parameters are also analyzed, and the robustness of this mechanism is verified on different networks.
Collapse
Affiliation(s)
- Litong Fan
- School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China
| | - Zhao Song
- School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China
| | - Lu Wang
- School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China
| | - Yang Liu
- School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China
| | - Zhen Wang
- School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China
| |
Collapse
|
3
|
Shi J, Liu J, Perc M, Deng Z, Wang Z. Neighborhood size effects on the evolution of cooperation under myopic dynamics. CHAOS (WOODBURY, N.Y.) 2021; 31:123113. [PMID: 34972342 DOI: 10.1063/5.0073632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 11/23/2021] [Indexed: 06/14/2023]
Abstract
We study the evolution of cooperation in 2×2 social dilemma games in which players are located on a two-dimensional square lattice. During the evolution, each player modifies her strategy by means of myopic update dynamic to maximize her payoff while composing neighborhoods of different sizes, which are characterized by the corresponding radius, r. An investigation of the sublattice-ordered spatial structure for different values of r reveals that some patterns formed by cooperators and defectors can help the former to survive, even under untoward conditions. In contrast to individuals who resist the invasion of defectors by forming clusters due to network reciprocity, innovators spontaneously organize a socially divisive structure that provides strong support for the evolution of cooperation and advances better social systems.
Collapse
Affiliation(s)
- Juan Shi
- School of Automation, Northwestern Polytechnical University, Shaanxi 710072, China
| | - Jinzhuo Liu
- School of Software, Yunnan University, Kunming, Yunnan 650504, China
| | - Matjaž Perc
- Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, 2000 Maribor, Slovenia
| | - Zhenghong Deng
- School of Automation, Northwestern Polytechnical University, Shaanxi 710072, China
| | - Zhen Wang
- School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Shaanxi 710072, China
| |
Collapse
|
4
|
Takács K, Gross J, Testori M, Letina S, Kenny AR, Power EA, Wittek RPM. Networks of reliable reputations and cooperation: a review. Philos Trans R Soc Lond B Biol Sci 2021; 376:20200297. [PMID: 34601917 PMCID: PMC8487750 DOI: 10.1098/rstb.2020.0297] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
Reputation has been shown to provide an informal solution to the problem of cooperation in human societies. After reviewing models that connect reputations and cooperation, we address how reputation results from information exchange embedded in a social network that changes endogenously itself. Theoretical studies highlight that network topologies have different effects on the extent of cooperation, since they can foster or hinder the flow of reputational information. Subsequently, we review models and empirical studies that intend to grasp the coevolution of reputations, cooperation and social networks. We identify open questions in the literature concerning how networks affect the accuracy of reputations, the honesty of shared information and the spread of reputational information. Certain network topologies may facilitate biased beliefs and intergroup competition or in-group identity formation that could lead to high cooperation within but conflicts between different subgroups of a network. Our review covers theoretical, experimental and field studies across various disciplines that target these questions and could explain how the dynamics of interactions and reputations help or prevent the establishment and sustainability of cooperation in small- and large-scale societies. This article is part of the theme issue ‘The language of cooperation: reputation and honest signalling’.
Collapse
Affiliation(s)
- Károly Takács
- The Institute for Analytical Sociology, Linköping University, 601 74 Norrköping, Sweden.,Computational Social Science-Research Center for Educational and Network Studies (CSS-RECENS), Centre for Social Sciences, Tóth Kálmán u. 4., 1097 Budapest, Hungary
| | - Jörg Gross
- Institute of Psychology, Leiden University, Wassenaarseweg 52, 2333 AK, Leiden, The Netherlands
| | - Martina Testori
- Organization Sciences, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands
| | - Srebrenka Letina
- The Institute for Analytical Sociology, Linköping University, 601 74 Norrköping, Sweden.,Institute of Health and Wellbeing, MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Berkeley Square, 99 Berkeley Street, Glasgow G3 7HR, UK
| | - Adam R Kenny
- Institute of Cognitive and Evolutionary Anthropology, University of Oxford, 64 Banbury Road, Oxford OX2 6PN, UK.,Calleva Research Centre for Evolution and Human Sciences, Magdalen College, High Street, Oxford OX1 4AU, UK
| | - Eleanor A Power
- Department of Methodology, The London School of Economics and Political Science, Houghton Street, London WC2A 2AE, UK
| | - Rafael P M Wittek
- Department of Sociology, University of Groningen, Grote Rozenstraat 31, 9712 TG Groningen, The Netherlands
| |
Collapse
|
5
|
Amaral MA, Javarone MA. Strategy equilibrium in dilemma games with off-diagonal payoff perturbations. Phys Rev E 2020; 101:062309. [PMID: 32688499 DOI: 10.1103/physreve.101.062309] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2020] [Accepted: 06/03/2020] [Indexed: 06/11/2023]
Abstract
We analyze the strategy equilibrium of dilemma games considering a payoff matrix affected by small and random perturbations on the off-diagonal. Notably, a recent work [Proc. R. Soc. A 476, 20200116 (2020)1364-502110.1098/rspa.2020.0116] reported that while cooperation is sustained by perturbations acting on the main diagonal, a less clear scenario emerges when perturbations act on the off-diagonal. Thus, the second case represents the core of this investigation, aimed at completing the description of the effects that payoff perturbations have on the dynamics of evolutionary games. Our results, achieved by analyzing the proposed model under a variety of configurations as different update rules, suggest that off-diagonal perturbations actually constitute a nontrivial form of noise. In particular, the most interesting effects are detected near the phase transition, as perturbations tend to move the strategy distribution towards nonordered states of equilibrium, supporting cooperation when defection is pervading the population, and supporting defection in the opposite case. To conclude, we identified a form of noise that, under controlled conditions, could be used to enhance cooperation and greatly delay its extinction.
Collapse
Affiliation(s)
- Marco A Amaral
- Instituto de Humanidades, Artes e Ciências, Universidade Federal do Sul da Bahia-BA, 45996-108, Brazil
| | - Marco A Javarone
- Department of Mathematics, University College London, London WC1E 6BT, United Kingdom
| |
Collapse
|
6
|
Seasonal payoff variations and the evolution of cooperation in social dilemmas. Sci Rep 2019; 9:12575. [PMID: 31467364 PMCID: PMC6715707 DOI: 10.1038/s41598-019-49075-3] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2019] [Accepted: 08/19/2019] [Indexed: 11/14/2022] Open
Abstract
Varying environmental conditions affect relations between interacting individuals in social dilemmas, thus affecting also the evolution of cooperation. Oftentimes these environmental variations are seasonal and can therefore be mathematically described as periodic changes. Accordingly, we here study how periodic shifts between different manifestations of social dilemmas affect cooperation. We observe a non-trivial interplay between the inherent spatiotemporal dynamics that characterizes the spreading of cooperation in a particular social dilemma type and the frequency of payoff changes. In particular, we show that periodic changes between two available games with global ordering best be fast, while periodic changes between global and local ordering games best be slow for cooperation to thrive. We also show that the frequency of periodic changes between two local ordering social dilemmas is irrelevant, because then the process is fast and simply the average cooperation level of the two is returned. The structure of the interaction network plays an important role too in that lattices promote local ordering, whilst random graphs hinder the formation of compact cooperative clusters. Conversely, for local ordering the regular structure of the interaction network is only marginally relevant as role-separating checkerboard patterns do not rely on long-range order.
Collapse
|
7
|
Wang X, Gu C, Zhao J, Quan J. Evolutionary game dynamics of combining the imitation and aspiration-driven update rules. Phys Rev E 2019; 100:022411. [PMID: 31574646 DOI: 10.1103/physreve.100.022411] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2019] [Indexed: 06/10/2023]
Abstract
So far, most studies on evolutionary game dynamics in finite populations have concentrated on a single update rule. However, given the impacts of the environment and individual cognition, individuals may use different update rules to change their current strategies. In light of this, the current paper reports on a study that constructed a mixed stochastic evolutionary game dynamic by combining the imitation and aspiration-driven update processes. The target was to clarify the influences of the aspiration-driven process on the evolution of the level of cooperation by considering the behavior of a population in which individuals have two strategies available: cooperation and defection. Through a numerical analysis of unstructured populations and simulation analyses of structured populations and of the random-matching model, the following results were found. First, the mean fraction of cooperators varied alongside the probability with which the individual adopted the aspiration-driven update rule. In the Prisoner's Dilemma and coexistence games, the aspiration-driven update process promoted cooperation in the well-mixed population but inhibited it in structured ones and the random-matching model; however, in the coordination game, the aspiration-driven update process was seen to exert the opposite effect on cooperation by inhibiting the latter in a homogeneously mixed population but promoting it in structured ones and in the random-matching model. Second, the mean fraction of cooperators changed with the aspiration level in the differently structured populations and random-matching model, and there appeared a phase transition point. Third, the evolutionary characteristics of the mean fraction of cooperators maintained robustness in the differently structured populations and random-matching model. These results extend evolutionary game theory.
Collapse
Affiliation(s)
- Xianjia Wang
- Economics and Management School, Wuhan University, Wuhan 430072, China
- Institute of Systems Engineering, Wuhan University, Wuhan 430072, China
| | - Cuiling Gu
- Institute of Systems Engineering, Wuhan University, Wuhan 430072, China
| | - Jinhua Zhao
- Economics and Management School, Wuhan University, Wuhan 430072, China
| | - Ji Quan
- School of Management, Wuhan University of Technology, Wuhan 430070, China
| |
Collapse
|
8
|
Takesue H. Roles of mutation rate and co-existence of multiple strategy updating rules in evolutionary prisoner's dilemma games. ACTA ACUST UNITED AC 2019. [DOI: 10.1209/0295-5075/126/58001] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
9
|
Fang Y, Benko TP, Perc M, Xu H, Tan Q. Synergistic third-party rewarding and punishment in the public goods game. Proc Math Phys Eng Sci 2019; 475:20190349. [PMID: 31423104 PMCID: PMC6694311 DOI: 10.1098/rspa.2019.0349] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2019] [Accepted: 06/18/2019] [Indexed: 11/12/2022] Open
Abstract
We study the evolution of cooperation in the spatial public goods game in the presence of third-party rewarding and punishment. The third party executes public intervention, punishing groups where cooperation is weak and rewarding groups where cooperation is strong. We consider four different scenarios to determine what works best for cooperation, in particular, neither rewarding nor punishment, only rewarding, only punishment or both rewarding and punishment. We observe strong synergistic effects when rewarding and punishment are simultaneously applied, which are absent if neither of the two incentives or just each individual incentive is applied by the third party. We find that public cooperation can be sustained at comparatively low third-party costs under adverse conditions, which is impossible if just positive or negative incentives are applied. We also examine the impact of defection tolerance and application frequency, showing that the higher the tolerance and the frequency of rewarding and punishment, the more cooperation thrives. Phase diagrams and characteristic spatial distributions of strategies are presented to corroborate these results, which will hopefully prove useful for more efficient public policies in support of cooperation in social dilemmas.
Collapse
Affiliation(s)
- Yinhai Fang
- College of Economics and Management, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, People's Republic of China
| | - Tina P. Benko
- Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, 2000 Maribor, Slovenia
| | - Matjaž Perc
- Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, 2000 Maribor, Slovenia
- Complexity Science Hub Vienna, Josefstädterstraße 39, 1080 Vienna, Austria
| | - Haiyan Xu
- College of Economics and Management, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, People's Republic of China
| | - Qingmei Tan
- College of Economics and Management, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, People's Republic of China
| |
Collapse
|
10
|
Wu Y, Zhang Z, Chang S. Heterogeneous indirect reciprocity promotes the evolution of cooperation in structured populations. CHAOS (WOODBURY, N.Y.) 2018; 28:123108. [PMID: 30599534 DOI: 10.1063/1.5066589] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/16/2018] [Accepted: 11/19/2018] [Indexed: 06/09/2023]
Abstract
A fundamental aspect of all biological system is cooperation. Human society is based to a large extent on mechanisms that favor cooperation. Indirect reciprocity is a crucial factor for the emergence and maintenance of cooperation in evolutionary games. In this work, we introduce a mechanism of heterogeneous indirect reciprocity in the prisoner's dilemma game, where an altruistic attribute is considered. The so-called altruistic attribute refers to the trait that when an altruistic individual cooperates, its neighbors, regardless of their strategies, can gain additional benefits. Intuitively, altruistic cooperative agents seem to do only what is beneficial for others, but in fact, their neighbors tend to cooperate in order to maintain the cooperative strategies of altruistic individuals. In this way, the neighbors of cooperative altruistic individuals not only guarantee their own additional benefits, but also indirectly protect the income of altruistic individuals, which strengthens the links between cooperative individuals, thus promoting the evolution of cooperation. The robustness of the results is verified on homogeneous and heterogeneous networks. Moreover, the results of individual heterogeneity corroborate the existing evidence that heterogeneity, almost irrespective of its origin, promotes cooperative actions. Our conclusions might provide additional insights into understanding the roots of cooperation in social systems.
Collapse
Affiliation(s)
- Yu'e Wu
- Coordinated Innovation Center for Computable Modeling in Management Science, Tianjin University of Finance and Economics, Tianjin 300222, China
| | - Zhipeng Zhang
- Coordinated Innovation Center for Computable Modeling in Management Science, Tianjin University of Finance and Economics, Tianjin 300222, China
| | - Shuhua Chang
- Coordinated Innovation Center for Computable Modeling in Management Science, Tianjin University of Finance and Economics, Tianjin 300222, China
| |
Collapse
|
11
|
Amaral MA, Javarone MA. Heterogeneous update mechanisms in evolutionary games: Mixing innovative and imitative dynamics. Phys Rev E 2018; 97:042305. [PMID: 29758674 DOI: 10.1103/physreve.97.042305] [Citation(s) in RCA: 59] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2017] [Indexed: 11/07/2022]
Abstract
Innovation and evolution are two processes of paramount relevance for social and biological systems. In general, the former allows the introduction of elements of novelty, while the latter is responsible for the motion of a system in its phase space. Often, these processes are strongly related, since an innovation can trigger the evolution, and the latter can provide the optimal conditions for the emergence of innovations. Both processes can be studied by using the framework of evolutionary game theory, where evolution constitutes an intrinsic mechanism. At the same time, the concept of innovation requires an opportune mathematical representation. Notably, innovation can be modeled as a strategy, or it can constitute the underlying mechanism that allows agents to change strategy. Here, we analyze the second case, investigating the behavior of a heterogeneous population, composed of imitative and innovative agents. Imitative agents change strategy only by imitating that of their neighbors, whereas innovative ones change strategy without the need for a copying source. The proposed model is analyzed by means of analytical calculations and numerical simulations in different topologies. Remarkably, results indicate that the mixing of mechanisms can be detrimental to cooperation near phase transitions. In those regions, the spatial reciprocity from imitative mechanisms is destroyed by innovative agents, leading to the downfall of cooperation. Our investigation sheds some light on the complex dynamics emerging from the heterogeneity of strategy revision methods, highlighting the role of innovation in evolutionary games.
Collapse
Affiliation(s)
| | - Marco Alberto Javarone
- School of Computing, University of Kent, Chatham Maritime, United Kingdom.,nChain Ltd., London W1W 8AP, United Kingdom.,School of Computer Science, University of Hertfordshire, Hatfield AL10 9AB, United Kingdom
| |
Collapse
|