Nothing better to do? Environment quality and the evolution of cooperation by partner choice

Ecology and the evolution of cooperation by partner choice and reciprocity

Organisms help each other to get resources, protection from enemies, and other goods, but not as much as would be best for their population. Partner choice, direct reciprocity and indirect reciprocity foster cooperation and help to align individual interests with the social good. However, we still do not know what ecological variables affect their success and interaction. I simulated the evolution of partner choice, direct reciprocity and indirect reciprocity, and the production of a good that partners donate to each other. I show that, with few exceptions, partner choice evolves whenever there is an initial social dilemma, under a wider range of conditions than direct reciprocity. Direct reciprocity is deleterious when the shared good is highly essential or very influential. Both partner choice and direct reciprocity compel individuals to produce close to the socially optimal quantity of the shared good. Direct reciprocity does so even when it is deleterious and reciprocators are rare. Indirect reciprocity succeeds when individuals can also choose partners, and in most cases contributes less than partner choice or direct reciprocity to alleviating social dilemmas. Partner choice may have allowed humans to use a set of collectively produced goods including clothing, fire, hunting tools, housing, and shared knowledge, to the point that they became essential.

Signalling and social learning in swarms of robots

This paper investigates the role of communication in improving coordination within robot swarms, focusing on a paradigm where learning and execution occur simultaneously in a decentralized manner. We highlight the role communication can play in addressing the credit assignment problem (individual contribution to the overall performance), and how it can be influenced by it. We propose a taxonomy of existing and future works on communication, focusing on information selection and physical abstraction as principal axes for classification: from low-level lossless compression with raw signal extraction and processing to high-level lossy compression with structured communication models. The paper reviews current research from evolutionary robotics, multi-agent (deep) reinforcement learning, language models and biophysics models to outline the challenges and opportunities of communication in a collective of robots that continuously learn from one another through local message exchanges, illustrating a form of social learning.This article is part of the theme issue 'The road forward with swarm systems'.

Scale at the interface of spatial and social ecology

Animals simultaneously navigate spatial and social environments, and their decision-making with respect to those environments constitutes their spatial (e.g. habitat selection) and social (e.g. conspecific associations) phenotypes. The spatial-social interface is a recently introduced conceptual framework linking these components of spatial and social ecology. The spatial-social interface is inherently scale-dependent, yet it has not been integrated with the rich body of literature on ecological scale. Here, we develop a conceptual connection between the spatial-social interface and ecological scale. We propose three key innovations that incrementally build upon each other. First, the use-availability framework that underpins a large body of literature in behavioural ecology can be used in analogy to the phenotype-environment nomenclature and is transferable across the spatial and social realms. Second, both spatial and social phenotypes are hierarchical, with nested components that are linked via constraints-from the top down-or emergent properties-from the bottom up. Finally, in both the spatial and social realms, the definitions of environment and phenotype depend on the focal scale of inquiry. These conceptual innovations cast our understanding of the relationships between social and spatial dimensions of animal ecology in a new light, allowing a more holistic understanding and clearer hypothesis development for animal behaviour. This article is part of the theme issue 'The spatial-social interface: a theoretical and empirical integration'.

Evolutionary dynamics under partner preferences

The fact that people often have preference rankings for their partners is a distinctive aspect of human behavior. Little is known, however, about how this talent as a powerful force shapes human behavioral traits, including those which should not have been favored by selection, such as cooperation in social dilemma situations. Here we propose a dynamic model in which network-structured individuals can switch their interaction partners within neighborhoods based on their preferences. For the partner switching, we propose two interruption regimes: dictatorial regime and negotiating regime. In the dictatorial regime, focal individuals are able to suspend interactions out of preferences unilaterally. In the negotiating regime, either focal individuals or the associated partners agree to suspend, then these interactions can be successfully suspended. We investigate the evolution of cooperation under both preference-driven partner switching regimes in the context of both the weakened variant of the donation game and the standard one. Specifically, we theoretically approximate the critical conditions for cooperation to be favored by weak selection in the weakened donation game where cooperators bear a unit cost to provide a benefit for each active neighbor and simulate the evolutionary dynamics of cooperation in the standard donation game to test the robustness of the analytical results. Under dictatorial regime, selection of cooperation becomes harder when individuals have preferences for either cooperator or defector partners, implying that the expulsion of defectors by cooperators is overwhelmed by the chasing of defectors towards cooperators. Under negotiating regime, both preferences for cooperator and defector partners can significantly favor the evolution of cooperation, yet underlying mechanisms differ greatly. For preferences over cooperator partners, cooperator-cooperator interaction relationships are reinforced and the associated mutual reciprocity can resist and assimilate defectors. For preferences over defector partners, defector-defector interaction relationships are anchored, weakening defectors' exploitation over cooperators. Cooperators are thus offered much time space to interact among cospecies and spread. Our work may help better understand the critical role of preference-based adaptive partner switching in promoting the evolution of cooperation.

Policy search with rare significant events: Choosing the right partner to cooperate with

This paper focuses on a class of reinforcement learning problems where significant events are rare and limited to a single positive reward per episode. A typical example is that of an agent who has to choose a partner to cooperate with, while a large number of partners are simply not interested in cooperating, regardless of what the agent has to offer. We address this problem in a continuous state and action space with two different kinds of search methods: a gradient policy search method and a direct policy search method using an evolution strategy. We show that when significant events are rare, gradient information is also scarce, making it difficult for policy gradient search methods to find an optimal policy, with or without a deep neural architecture. On the other hand, we show that direct policy search methods are invariant to the rarity of significant events, which is yet another confirmation of the unique role evolutionary algorithms has to play as a reinforcement learning method.

Social learning in swarm robotics

In this paper, we present an implementation of social learning for swarm robotics. We consider social learning as a distributed online reinforcement learning method applied to a collective of robots where sensing, acting and coordination are performed on a local basis. While some issues are specific to artificial systems, such as the general objective of learning efficient (and ideally, optimal) behavioural strategies to fulfill a task defined by a supervisor, some other issues are shared with social learning in natural systems. We discuss some of these issues, paving the way towards cumulative cultural evolution in robot swarms, which could enable complex social organization necessary to achieve challenging robotic tasks. This article is part of a discussion meeting issue 'The emergence of collective knowledge and cumulative culture in animals, humans and machines'.